CISA3 Researchers Look Into the Past with High-Resolution Digital Scans of Italy's Palazzo Medici

(PhysOrg.com) -- The Palazzo Medici, one of most significant landmarks of the Italian Renaissance, has always been something of a touchstone — it was completed in 1460 for the wealthy Medici family and later served as the model for Florence's more ornate Pitti and Strozzi palaces. It's fitting, then, that the palace (now called the Palazzo Medici-Riccardi) has also become a prototype for researchers pursuing high-resolution digital cultural forensics, a field being pioneered by scientists at UC San Diego's Center of Interdisciplinary Science for Art, Architecture and Archaeology (CISA3).

"The Palazzo Medici is home to untold stories and engineering feats," says CISA3 Director Maurizio Seracini. "It was the birthplace of the Renaissance, the house of the Medici family. As such, the building and the Medici family, in particular, shaped the city of Florence and left a legacy in Italy that has been unparalleled since. But the Palazzo is being renovated, and will never be exactly the same, so this is our chance to conduct a very detailed, high-resolution multi-spectral scan of the building, supporting 3-D modeling and visualization to determine what else might be behind those walls and floors that isn't visible yet.

"Once we have a digital record of the Palazzo Medici," he continues, "we can refer back to the multi-spectral image data and gain a better understanding of the way the building was constructed, how it was altered and what its current state of health is. So, as you can see, this project nicely blends the engineering, architecture, arts and science worlds into one embodiment."

The research is funded through a $75,000 UCSD Chancellor's Interdisciplinary Collaboratories Grant, with additional support from the UCSD division of the California Institute for Information Technology (Calit2), where CISA3 is based. Funding is also provided by several private donors, including Bob and Natalie Englekirk, Sandra Timmons and Rick Sandstrom and Paul and Stacy Jacobs (some of whom visited the Palazzo and saw the work in progress).

The Palazzo Medici project is one outcome of a Memorandum of Understanding signed in 2007 by Seracini, Province of Florence's President Matteo Renzi, Calit2-UCSD Director Ramesh Rao, UCSD Jacobs School of Engineering Dean Frieder Seible and former UCSD Division of Arts & Humanities Dean Michael Bernstein.

"Our goal is to make the history of the Palazzo known, from an artistic and architectural point of view," Renzi says. "It is anticipated that these activities will require the use of highly advanced technologies in the image diagnostics area, followed by design and development of new methods that permit the location and detection of hidden mural paintings."

As part of a related CISA3 project, Seracini, who is a native of Florence, conducted thermal imaging of the facade of the Palazzo Medici to determine, via differences in heat signatures, where doors or windows might have been covered over. He's also using multispectral imaging in an attempt to locate Leonardo da Vinci's lost "Battle of Anghiari" painting in Florence's Palazzo Vecchio — technology that will eventually be employed at the Palazzo Medici. So far, research of the Palazzo Medici reveals that it sits upon a network of underground tunnels and hidden passageways.

"Through Maurizio and his work, we have the opportunity to gain access to these very famous buildings. We decided to start with two of the most significant buildings and expand from there once we've established a baseline methodology," says Falko Kuester, professor of visualization and virtual reality at the UCSD division of Calit2.

"The thermal scans that Maurizio has already taken have more than just an "aha" meaning — they signify that structural characteristics have changed over time," he continued. When you're changing the stiffness of a wall, for example, you're also changing the behavior of the wall during a seismic event. If later, we want to model a part of the building and determine how well it will perform during the next seismic event, we need to be able to describe it fundamentally at the most detailed level possible. One benefit of this methodology is that it provides a very complete three-dimensional model as well as volumetric composition, which allows us to refine the model further."

Once the geometry of the building is established through 3-D scanning, the team will return to the Palazzo to conduct further thermographical scanning to see where renovations were made in the past. Another goal of the project is to determine any connections between the Palazzo and surrounding buildings and conduct a critical reading and interpretation of the extant collection of letters of the Medici family.

Joining Kuester and Seracini on the project are graduate student researchers Michael Olsen, Kevin Ponto, Jason Kimball, So Yamaoka and Vid Petrovic of UCSD's Departments of Computer Science and Engineering, as well as Structural Engineering. Olsen, Ponto and Kimball recently spent a month at the Palazzo Medici compiling a very detailed terrestrial laser scan of some of the building's key architectural features, and will return at various times over the next year to complete the 3-D reference model.

The researchers say the data-gathering process can be tedious and time-consuming, especially in the initial stages when they must set up reference markers to help align the scans, much as a photographer would when creating a 360-degree panoramic shot. "If the markers are off by two centimeters," Ponto explains, "then the next scan is off by two centimeters. We try to find cracks in the flooring to mark the spots we're scanning from, and we also take photos and keep a journal to mark our locations."

Adds Kimball: "There's a lot of strategy involved in placing the targets and the scanner. Sometimes, we spend up to an hour just creating the strategy for our approach."

"People might think: Why not just take a lot of photos of the building? But with photos, you have problems with lens distortion," Ponto points out, "and you're always going to miss a spot."

Once the markers are in the proper place, the Leica ScanStation2 laser scanner takes anywhere from 10 to 45 minutes to scan 180 degrees in one direction, depending on the desired resolution. Selected areas of the palace — which takes up an entire city block — are being scanned at a resolution of one millimeter. Once visualized on Calit2's 286-pixel HIPerSpace display wall, the digital model parallels the quality of photographs.

"Overall, we're able to scan in the sub-centimeter range, which means we end up with hundreds of million to billions of points," Kuester notes. "To my knowledge, this type of scan has not done before at this resolution. In the end, what we have is not just a blueprint. A blueprint might be interpreted throughout construction, or be obsolete as a result of later modifications. This is a true representation, a digital embodiment of this particular structure. In fact, it's much more than just a structure: It's an artifact."

Such detailed scanning means the team must cope with a substantial amount of data — hundreds of gigabytes to terabytes worth. Ponto said the team, working with Petrovic, had to develop 3-D point cloud rendering software to be able to examine their data and look for gaps, because no program currently on the market could do the job.

"For this data to be usable, it needs to be synchronized so that different scans are consistent," Olsen adds. "To be able to visualize and interact with such a large database in real time has been a challenge. But the new technologies we're implementing enable structural engineers to obtain a wealth of information about buildings, so they are well worth the effort. This is invaluable to necessary upgrades or retrofits of buildings so that they can be preserved during damaging events.

"Currently, information about the structural integrity is obtained by destructive sampling and testing of the structure, or is estimating based on similar buildings," he continues. "The methods being implemented at the Palazzo Medici provide a new look at the structure without damaging it whatsoever."

Adds Kimball: "It's of amazing cultural importance that we can get a snapshot of what the Palazzo looked like in 2008. In 500 years, who knows how much it might change?"

What's unlikely to change is the popularity of the Palazzo Medici, now a museum that receives hundreds of visitors per day. "We got a lot of interesting, confused and curious looks from people while we're scanning," Kimball says.

CISA3 Assistant Director Alex Hubenko adds that the project has helped solidify CISA3's relationship with the government of the Province of Florence and has generated exposure for UCSD among the people of Florence.

"Our work in the Palazzo has helped the Province understand what we're doing and why we're doing it, and has the potential to lead to other collaborations in the future. Kevin, Jason and I even presented our results — in Italian — at a public presentation at Florence's annual Genio Fiorentino (the Genius of Florence) conference in May."

"Overall there seems to be a tremendous amount of excitement about the project among the people of Florence, which has given our students an interesting perspective on research," adds Kuester. "There has also been an amazing amount of media coverage of these projects — basically, the 'who's who' of printed and broadcast media have been following our research. On one hand that's very exciting, but it also takes a very careful balancing act to make sure the research gets advanced at the same time the media's questions are being answered."

"One of the greatest challenges are the visitors to the Palazzo Medici," he says. "It's an icon, and other people are taking in these beautiful places just as we are, but that can be tricky for us when we're scanning line-of-sight. It can't be avoided, and our approach is fundamentally designed around the idea that day-to-day operations at the selected site, are not disrupted by our efforts."

Adds Ponto: "The challenge isn't necessarily how big a room is, it's how many points we have to scan. We have had a lot of scans that captured people walking through the room. If people moved quickly through the laser beam, we would still get a few spurious points, but we could edit those out. The courtyard was pretty difficult to scan because there were all these rows of lemon trees" — lemon trees that inspired several renaissance artists, according to historical records of the palace.

Even color-correction plays a part in these incredibly detailed scans: Olsen is responsible for blending the scans so that those performed under different lighting conditions constitute a more consistent representation of the actual building.

"From many perspectives," Kuester adds, "this is actually one of the unique parts of the methodology that were working on. This is not a process that can be standardized. The only way to get the broad description of the building that we're seeking is to do so in the environment that we live in today. As soon as we lock the doors and move people out, broad application of the developed methodology to other artifacts will not work. We don't work in the clean lab environment anymore."

Olsen, who is training a group of Italian graduate students and researchers to assist with CISA3's work, adds that once the scans are complete, they will not only benefit the scientific and engineering communities but will serve as an unparalleled resource to the world-at-large.

"This project provides a way for people to enjoy and learn about the palace without having to travel to it," he says. "This makes the opportunity available to those who may never have the opportunity to go over there. Ideally, these models will be integrated into museums and schools so that as children and adults are learning about history, they can see these places and interact with them to enhance the learning experience."

Although the CISA3 researchers are among the first in the world to employ high-resolution, multi-spectral, 3-D digital scans to analyze historic structures, they point to several buildings they see as potential candidates for future work in the field.

"You pretty much can go down the list of monuments," Kuester laughs, "and the UNESCO World Heritage List is a wonderful reference for where we should go next. The exciting part is that this is actually doable, if we can raise the needed funds."

"But how do you measure success? One sign is when others start to mimic you, and that's what is beginning to happen with the methodologies and technologies that CISA3 is developing. And that's a good thing. Clearly Florence has been very significant throughout history and the history of art, and we'd like to one day create a 'Digital Florence.' There's work to be done for generations to come, and it's important work."

Provided by University of California, San Diego