There’s a lot of beauty to gaze upon in Don’t Worry Darling, and we’re not just talking about the film’s cast of young, hot Hollywood A-listers, some of whom started dating during filming and some of whom may are rumored to hate each other. While much of the conversation leading up to the film’s release focused on alleged behind-the-scenes drama between director Olivia Wilde and star Florence “Miss Flo” Pugh, leaked texts from one-time leading actor Shia LaBoeuf, Chris Pine’s pained face at the Venice Film Festival press conference, and more than a few less-than-glowing early reviews, even the film’s harsher critics agree that Olivia Wilde’s second directorial feature is visually impressive, chock-full of gorgeous Palm Springs vistas, swinging Eisenhower-era fashions, and more than a few excellent examples of midcentury architecture.

Chief among those architectural marvels is Richard Neutra’s Kaufmann Desert House, which the celebrated modernist architect built in 1946 for Edgar J. Kaufmann Sr. (the same Pittsburgh department store mogul that charged Frank Lloyd Wright with designing Fallingwater). The International Style structure plays off the nearby San Jacinto mountains using walls of windows that thoughtfully frame vistas and natural materials such as sandstone and birch-veneered plywood. Vertical aluminum louvers protect the various wings from harsh desert temperatures and create a strong connection between the inside and outdoors. Neutra novices might recognize the home’s adjacent poolside pavilion from photographer Slim Aarons’s quintessential 1970s photo, Poolside Gossip, which featured a couple of mod, bubble-haired socialite types perched on lounge chairs while in candid conversation. (A writer for the New York Times once said the image “has become as much a symbol of modernism” as its setting.) Olivia Wilde certainly did. As she told Variety, the director actually had a print of Aarons’s photo on her wall when she was beginning work on Don’t Worry Darling, and had visions of using the home as a model for the film’s set design.

But it wasn’t until she and production designer Katie Byron started to scout for locations in Palm Springs that location manager Chris Baugh told them they might actually be able to shoot there. Prior to Don’t Worry Darling, no Hollywood productions had been allowed to shoot in the architecturally significant residence, which is designated a Class 1 Historic Site by Palm Springs City Council. Baugh, however, had made headway with the home’s longtime owners, Brent and Beth Harris, through an acquaintance and thought he might be able to get Wilde and company in to at least see the space. (The Harrises, he a financial executive and she an architectural historian, bought the house in the early 1990s, but sold it this May to an undisclosed owner for a reported $13 million.)

In the film, the angular, glass-and-metal building is the residence of mysterious Victory svengali, Frank (Chris Pine) and his all-too-perfect wife Shelley (Gemma Chan). (Victory being the incredibly insular company town that everyone works, shops, and resides in throughout the film. The goal of Victory is to change the world, Pine’s character says, though it’s not clear until very late in the movie what he means by that.) Viewers first see the home when Jack (Styles) and Alice (Pugh) are invited to a garden party there, which is considered quite an honor. Baugh says he immediately thought of the Kaufmann House when hearing about Pine’s character, who, he explains in an interview, “has idealized affluence, taste, and wealth. When you’re thinking about where somebody like that might live in a community like Palm Springs, it’s got to be a pretty spectacular place.”

Once Wilde, Baugh, and Byron toured the Kaufmann House, they fell in love with the idea of using it for the scene. Baugh then embarked upon a months-long letter-writing campaign with the very thoughtful and protective owners, providing them with what he describes as “detailed plans about how we could shoot safely and what the project was.”

“It’s a very delicate, historic, fragile property,” says Baugh. “It was built with soft materials and it’s very easy to damage the home. If you bumped into the wall too hard you could scratch something. There were certain floors you couldn’t walk on unless you were barefoot because things were delicate. It just wasn't worth it to the owners to have a film crew in there [previously], and so that was a difficult discussion.”

“We didn’t want to be known as the people who destroyed the Kaufmann House.”

Still, Baugh did manage to convince the owners to let them use the Kaufmann House as a filming location; it’s actually where the first day of shooting on Don’t Worry Darling took place. In the run-up to the shoot, Byron and her team toured the home several more times to prepare, which she says led her to develop an even greater love of Neutra’s work. “When you prep a location, you spend a lot of time there,” Byron explains. “You keep going back and measuring details and looking at every single nook and cranny of the place in order to create plans for the art department. A lot of that time, we were also studying all of the details like every little piece of hardware.”

Byron says her team even fell in love with a paint color that was original to Neutra’s design. She describes it as “almost black, but like the deepest, darkest brown,” and says it could only be purchased through a special order. The production team decided to weave it into the whole film, connecting the rest of the interior sets back to Frank’s house in a way that’s both intentional and suggestive, creating what Byron calls “a little bit of noise throughout the film.”

When it came time to actually shoot at the Kaufmann House, tensions were understandably high. Film crews can contain about 150 people, many with dirty shoes, heavy equipment, and rogue tool belts. Because, as Baugh says, “We didn’t want to be known as the people who destroyed the Kaufmann House,” the production team took extra special precautions. Baugh says they enlisted a special company to install protective materials all over the property, using durable floor and edge coverings in every hallway and on every corner and door jamb. “We had people there the whole time from before the first people walked in until the last people left just to sit there and monitor things,” Baugh says. “The owners of the house also hired representatives to stand in each room quietly watching to try to prevent any accidents or rough behavior.”

Only certain production staff and cast members were even allowed in the home in an effort to keep the interior a tightly controlled environment. No camera dollies were permitted on-site either. “A dolly is almost like a mini train car. It’s a heavy piece of metal,” Baugh explains. “If that gets rolling and rams into a wall, you’re done. It’s going to damage that wall.”

All that reverence paid off, not just because Don’t Worry Darling’s first day of shooting went off without a hitch, but because the location helped the cast and crew better understand the spaces the film would inhabit, both production and storywise. “In order to shoot in the Kaufmann House, we had to come to set with the utmost respect,” says Byron. “Sometimes [on film sets] people kind of lose the plot and forget where they are. With the Kaufmann House, everyone was very aware of where we were, and that carried throughout the film. Anytime we were in any space, we wanted to treat it as if it was a celebrity itself.”

Baugh agrees, saying, “Everything about that house is spectacular. It’s what you feel when you’re walking through the hallway. It hits you at an emotional level that’s deeper than aesthetics; you’re feeling sensations that have a visceral impact. I can’t even describe it, but the Kaufmann House has it. It’s a very special space, and it was an honor to be there.”

Top photo courtesy of Warner Bros. Pictures.

How the Caterpillar House Set the Stage for Zendaya’s Secret Movie Filmed Under Lockdown