Although films and TV reveals love to point out the trope of the lonely genius, working alone to save lots of the world, in actuality collaboration is a necessity in an effort to understand probably the most unbelievable initiatives. And what higher option to discover these collaborations than via conferences? One notable instance is definitely proven by two of the keynote audio system at AMUG 2023, Nicholas Jacobson, a professor on the College of Colorado’s Anschutz Medical Campus, and Rob Ducey, a technical supervisor at animation specialist Laika Studios. The 2 met by probability at AMUG 2019, bonding over a little-known 3D modeling and printing course of known as bitmap printing. Now, they’re working collectively on a venture to make use of the expertise to create modern designs for pediatric epilepsy, cardiology and cleft palates. We sat down with the 2 to be taught extra.
3DN: Might you please introduce yourselves?
Nick: Hello, I’m Nick Jacobson and I’ve been concerned in 3D printing since 2004. My background is in structure, and I first labored on 3D printing in structure faculty utilizing binder jetting with refined corn starch on a Z-corp printer. We made panorama and architectural fashions and this was simply on the daybreak of the computational design increase in structure. College students we simply beginning to learn to code and program, so we grew to become very occupied with translating these concepts via digital fabrication to create uncommon varieties.
This love of kind technology adopted me all through my architectural profession, the place I labored for some nice architects resembling Renzo Piano, Zaha Hadid, and Harry Teague. After some years training structure, I used to be accepted into the Harvard Graduate Faculty of Design and the Harvard Enterprise Faculty. Right here, I used to be a part of an early analysis program trying into Voxel Printing on what was then Connex and now Stratasys. We developed software program to mannequin and 3D printing utilizing Voxel expertise. Initially the work centered on structural engineering and design.
At present, I work on the College of Colorado on the Anschutz Medical Campus and have a lab centered computation and 3D printing for medical purposes. The software program we developed at Harvard has confirmed to be beneficial for the medical world, just by modifying the enter information. It might appear to be a stretch but it surely’s similar to working with structural evaluation information. At present, we’re centered in 3 areas: Surgical steerage and training fashions, Prosthetics, and Implants… all revolving round comfortable tissue purposes.
Rob Ducey (left) and Nick Jacobson (proper)
Rob: Hi there, I’m Rob Ducey and I’m at the moment the technical supervisor for what we name speedy prototype at LAIKA. In a nutshell, our group handles modeling, facial animation, and fabrication for the advanced puppets which are the celebs of our movies. The moniker “speedy prototype” is a legacy of the improvements we pioneered on our first movie Coraline again in 2009 utilizing 3D printing for facial animation. This work acquired a sci-tech Oscar in 2016. I grew to become concerned with 3D printing on that venture again in 2006 when Brian Mclean and Martin Munier got here to the studio with the daring plan to make use of Objet polyjet printers to generate the 1000’s of faces required to create the facial performances of the characters in that movie. I used to be tasked with constructing a pipeline to mannequin, animate, and in the end print these faces. Ever since that preliminary manufacturing, we have now continued to innovate our processes to reap the benefits of and now push the boundaries of multi-material 3D printing. A humble positive artwork main, I now get to tinker with a few of the most cutting-edge expertise in pc aided fabrication.
3DN: You might be from two very completely different disciplines, how did you meet?
Nick: We met at AMUG a few years again when LAIKA was giving the keynote. Their presentation was superb and highlighted how they used bitmap printing expertise to create a lot of their work. It was cathartic to listen to others utilizing the identical esoteric expertise in an enormous approach. I used to be in awe of them and didn’t have the heart to talk with them eighth away. Nevertheless, afterward Brian and Rob Ducey got here to my speak on voxel modeling and bitmap printing for surgical procedure. Afterwards, they have been the primary two to return up and speak to me and we bonded over our shared admiration for bitmap printing and what’s doable. Then we linked just about and mentioned our particular person approaches to the expertise. It was instantly apparent that we had easy options to a few of the issues the opposite have been wanting to aim. For instance, the particular results world had lengthy had options to working with skinny hair like buildings, which made working with tractography a breeze, one thing we had been wrestling with for some time. In our first conferences, and nonetheless immediately, we rattle off tons of concepts that the particular results world has been doing for years that permit for developments in medical purposes, issues that nobody had via of earlier than as a result of there’s little to no overlap between the professions. All it took was for a easy connection to be made which may permit for these conversations.
Rob: Right here on the studio, we’re at all times on the hunt for brand spanking new concepts and technical developments we are able to make use of to increase our capabilities and redefine what is feasible in cease movement animation regardless of the place they arrive from. In our use of multi-material printing, we found a small group of like-minded tinkerers who had entry to options nonetheless thought of “in growth”. After some persistence and pestering, we have been capable of have conversations and visits with college researchers at establishments resembling MIT and Virginia Tech who have been inquisitive about what a bunch of artists and filmmakers in a warehouse within the rain-soaked northwest have been doing with these machines. These conversations turned us on to AMUG, which we first attended in 2015 after which once more in 2019 to present a keynote. When attending AMUG, we’re at all times laser centered on discovering others we are able to be taught from or just share our pleasure for multi-material 3D printing. I noticed the outline of Nick’s presentation and made certain we have been there. We approached Nick excited to seek out somebody in what was then a really small membership of “voxel printers”. Partly it was to point out off what we had realized, but in addition to see what secrets and techniques he had uncovered. It was like discovering somebody in a misplaced tribe.
Laika has used 3D printing for quite a lot of years for its movies (picture credit: Laika)
3DN: Might you inform us extra about your venture? Moreover, may you clarify bitmap printing?
Nick: Our initiatives have centered in 3 areas: 1. Advancing bitmap printing methods and workflows 2. Multimodal information fusion for medical purposes centered on comfortable tissue 3. Parametric design for prosthetics.
Bitmap printing is the power to print straight from a stack of bitmap photos. It’s very very similar to how an inkjet printer works, besides on this case the pages are layers. It’s quite simple and leverages how a polyjet printer natively works, we’re are primarily bypassing the inner slicer of the printer and sending our personal slices. These slices management the situation of each single droplet. Nevertheless, which means we have to outline every droplet and that’s more durable. Most, if not all, modeling software program used for 3D printing makes use of a floor mesh-based modeling paradigm which represents objects by their boundaries, ignoring inner volumetric information. In lots of purposes, resembling medical and structural, volumetric data is efficacious to have the ability to seize. So, in an effort to bitmap print, we use a special paradigm for modeling known as ‘voxel modeling’ which fashions volumetric information. The outcome permits us to straight translate image-based information, resembling DICOM information, right into a 3D printed object with little to no lack of information. The opposite main good thing about voxel modeling and bitmap printing is the power to manufacture a gradient of fabric by controlling droplet mixing ratios via dithering.
- At present, the key downside with bitmap printing is that in an effort to do it, you must primarily create the software program to create the information your self. There aren’t any off the self-workflows which permit somebody one to easily create a bitmap print. So, we have now been working to mix our completely different workflows and create new workflows which assist us to raised perceive and tune our skill to create bitmap prints. There are various components which have to be considered and which nonetheless have to be labored out. Materials mixing of mechanical and colour properties are nonetheless arduous to foretell and we regularly have surprising outcomes. That is due to the sheer variety of calculations a pc must do to digitally preview and predict how a print will end up. Our main focus is geared towards quantifying colour based mostly mixing ratios with translucency and qualifying mechanical properties as they differ throughout a gradient.
- Multimodal information fusion is an inherently easy a part of voxel-based modeling. Voxel modeling makes use of compositing as an alternative of booleaning, which makes the combining of quite a few advanced information sources easy and computationally cheap. Moreover, voxel modeling permits for the power to volumetrically change colour with out modifying the shape. These components permit for us to mix many information sources, each bitmap and vector based mostly, in a single mannequin resembling MRI, fMRI, PET, MEG, Tractography, and implanted units. Our work has centered on growing the workflows which permit us to manage every information supply individually whereas combing all of the sources right into a single holistic mannequin.
- Parametric design is a quite common modeling method in each structure and is the ‘bread and butter’ of cease movement animation. Now we have been working to create ‘rigged’ modeling software program which permits for the facile manipulation of advanced varieties over time. Our focus with this work has been geared towards designing affected person particular prosthetics which have to be modified over time. For this work we have now mixed medical information with commonplace cease movement animation software program to primarily create a cease movement collection of fashions for human prosthetics.
The venture may assist deal with pediatric epilepsy (picture credit: Nicholas Jacobson and Robert Ducey)
Rob: After our preliminary assembly it grew to become clear that there was rather a lot to be taught from one another about how one can design volumetrically for printing. We acknowledged some inherent qualities of the medical imaging course of that leant itself to volumetric information. On the studio we had been growing methods of taking volumetric information from a visible results workflow and changing that to multi-material printing. The important thing to that course of was “bitmap printing” or, because it has now come to be often called “voxel printing”. Up till this level we had solely involved ourselves with the outside look of a printed object and the skin layers that affected this. Our partnership with Fraunhofer and their product Cuttlefish and was important to mastering that. This gave us a familiarity and entry to printing from slices, which is basically what bitmap printing or voxel printing is. We engineered a course of to creator slices from our volumetric information that we may print. Nick’s work was particularly attention-grabbing within the varied modalities of medical imaging he was working with in DICOM format. These have been straight analogous to slices and have been fascinating to see in printed kind. We knew there may very well be some apparent methods to combine that information right into a VFX pipeline and generate some cool trying prints. From there we started discussing methods to feed that again into the areas of analysis Nick and the group on the College of Colorado have been exploring. This led to different discussions of leveraging methods and strategies effectively explored for animation and visible results that we would apply to ongoing medical analysis. A singular crossover for LAIKA is the truth that we require related precision, scales, and ranges of element within the components we engineer and generate that you simply may count on in medical purposes. Moreover, we convey some insights about how one can make one thing visually compelling.
3DN: What purposes do you envision for this?
Nick: We envision many purposes for this and have solely simply scratched the floor! For now, we’re centered on two areas: Pediatric Epilepsy Surgical procedure and Cleft Palate. Pediatric Epilepsy Surgical procedure is probably probably the most highly effective utility of bitmap printing for presurgical planning and intraoperative steerage. It’s because epilepsy is generally physiologically based mostly and in an effort to plan for therapy surgeons overview many 2D based mostly volumetric information sources, individually and sequentially, to kind a psychological understanding of the affected person. We’re specializing in combining the quite a few, and rising, obtainable datasets into one holistic 3D mannequin for use for planning and steerage. We plan to begin a potential trial earlier than the tip of the 12 months to judge the effectiveness of getting such a mannequin for therapy planning and publish operative outcomes.
Our work with cleft lip and palate is concentrated on a presurgical method known as Naso Alveolar Molding. It is a frequent therapy for infants born with cleft lip and palate that begins nearly instantly after start. The therapy includes putting an acrylic gadget onto the palate which directs development with the aim of closing the cleft. The outcome eliminates a surgical procedure and makes all subsequent surgical procedures extra profitable. Nevertheless, changes have to be made weekly, requiring households to go to the clinic weekly for 3-9 months. That is arduous, even should you reside shut. Nevertheless, there are few suppliers and entry to this therapy is prohibitive to the overwhelming majority of sufferers. So, our work is concentrated on offering a 3D printed answer which might permit for affected person to obtain a set of fashions within the mail, eliminating the necessity for weekly workplace visits. This could permit for anybody with the power to take a dental impression the power to obtain this therapy.
Rob: Nick and his group are driving the analysis about how one can apply what we’re studying within the discipline of drugs. For the studio, this has helped make clear each how we are able to technically design utilizing voxels and about how biology designs kind and performance volumetrically from the angle of medical specialists. Making use of these ideas of biology to design is one thing additive processes are uniquely poised to take advantage of.
The method may assist with Naso Alveolar Molding for cleft palate sufferers, a situation whereby holes are current at start within the palate (comfortable and arduous) or within the lip (picture credit: Dawn Orthodontics)
3DN: Any final phrases for our readers?
Each: If you want to be taught extra concerning the venture or about us generally, be sure you attend AMUG 2023. We will likely be giving the keynote speech, “Collaborations Between an Animator, an Architect, and a Surgeon: The Keys to Impactful Innovation in Medication” on Tuesday, March twenty first. If you happen to haven’t already, you possibly can register for the convention HERE.
What do you assume the work that Nick and Rob are doing? Do you assume 3D printing conferences play a key position in encouraging collaboration within the business? Tell us in a remark beneath or on our LinkedIn, Fb, and Twitter pages! Don’t neglect to join our free weekly E-newsletter right here, the newest 3D printing information straight to your inbox! You can too discover all our movies on our YouTube channel.
*Cowl Picture Credit: Nicholas Jacobson and Robert Ducey