“We have a team working on CI, there’s a team managing vCenter, and then there’s a team working on Mac hardware in the data center. “If you think about the entire system, it’s not actually one team that’s managing everything,” said Ruan. “Because then you’re questioning, ‘is my test actually flaky or is it the infrastructure’? So it’s really important to have reliable infrastructure that you can trust.”ĭropbox didn’t have a tremendous amount of expertise in-house for running VMware on Mac computers, and there were multiple teams involved in the process which added even more complexity. “It’s confusing when you can’t trust the underlying infrastructure,” said Ruan. When a test failed, there was some lingering doubt as to whether it was the infrastructure or the code that was causing the failure. The ESXi service crashing for an unknown reason didn’t inspire a lot of confidence in their staff developers. That made coordination between the teams a challenge,” said Ruan.
We didn’t know why, but it was causing us a lot of headaches. “One of the issues we saw was a lot of ESXi crashes on the Macs. While their CI pipeline worked very well, Dropbox started running into problems as they increasingly added more hosts to their cluster. “If you have VMs that live for minutes only, having fast clone time is pretty important.” Growing Pains
“These are very short-lived VMs,” said Ruan. VMware’s Linked Clones feature allows VMs to act more like containers, which was important to Dropbox as their VMs only live for minutes or hours. “We run VMs using the hosts’ SSDs, and we use linked clones to get the templates off of the data stores,” said Ruan. They used shared data stores to share VM templates across the hosts. For example, if they wanted to limit iOS workloads to a specific set of Mac hosts. The Dropbox team used the clusters to isolate work and control how many resources each workload could use.
There’s also the fact that we support multiple OS versions, so we have to run end-to-end tests on multiple OS X versions.”ĭropbox was running a single VMware vCenter with hundreds of Mac hosts, organized into multiple clusters. The more features you have, the more tests you have. And then there’s the number of features and tests. The more engineers you have, the more builds you run. A few factors contributed to Dropbox’s scalability issues, said Paul Ruan, a software engineer at Dropbox: “One is the number of engineers we have. This means reliable, scalable infrastructure for Mac and iOS app development is critical to the company’s success.Īlthough Dropbox’s Mac continuous integration (CI) operations ran well on an in-house VMware build cluster, the team eventually ran into problems as the size and scale of their development needs grew. Dropbox runs natively on Mac products like iPhones and MacBooks.
0 kommentar(er)
