Playing the central role in the creation of the biggest movie production in history, The Lord of The Rings trilogy, Weta Digital is world famous for making implausible fantasy stunningly real.

Putting the finishing touches on King Kong, a big challenge for Weta Digital was efficiently managing and protecting millions of epically proportioned data files, created by a team of digital artists.

Partnering with New Zealand IT and consulting company and NetApp storage solutions provider Fujitsu, Weta Digital has established a virtualised storage platform that can scale to variable demands, eliminating bottlenecks and hotspots. Significantly, it is one of the largest NetApp sites in Australasia.

Weta Digital has grown rapidly since its 1993 inception and first film project, the highly acclaimed Heavenly Creatures. From its early days of SGI server technology, Weta Digital has ramped up its computing and storage infrastructure, where today a compute farm of 4,500 processors and 220 terabyte (TB) of NetApp storage manages the workload created by up to 500 digital artists.

Weta Digital chief technology officer Milton Ngan explains the significance of storage: “Because we are so heavily technology-based, IT is a critical part of the business. Weta Digital required high performing and readily scaleable storage for artists to do their work.”

When you consider Weta Digital’s iterative production environment and the vast size and richness of its digital artistry, the mission critical nature of storage becomes apparent. For example, single image files range up to 100MB in size – not huge in the scope of enterprise IT, but consider that it takes 24 of these files to make just one second of film. 3D files are even bigger, with a single shot accounting for 1TB of storage. As Weta Digital has grown, the issue became performance and scalability.

The view shared by both Ngan and Fujitsu was that it was the role of storage to alleviate any possible bottleneck – storage was responsible for putting all the information online so that it was accessible to digital artists and delivered in a scalable manner to the compute farm.

Growing from just two NetApp filers and 400 processors, today four NetApp FAS3050 high-performance clusters form a unified NetApp architecture servicing Weta Digital’s compute farm.

Running on NetApp’s Data ONTAP microkernel operating system, Weta Digital is able to consolidate UNIX and Windows platforms in a central location. A smart Weta Digital-created software system, called DSMS, virtualises storage provided by 28 primary NetApp filer heads, so they can be viewed and managed as a single logical unit. Symlinks provide the virtualisation that allows Weta Digital to seamlessly migrate data between filers.

The allocations are extremely fast and can be done in parallel. What’s more, the system is able to manage quotas so that the appropriate volumes of data are allocated to the right filers.

With its unified NetApp architecture, Weta Digital gets swift, reliable access to data, thanks to a stable storage platform that can scale up to meet extremely heavy demands. In practical terms, this means Weta Digital is more efficient, because access to data is better, so it can work faster.

Enhanced scalability means any unit of work is now spread across all the NetApp filers. Whenever processors are added the new load is automatically rebalanced – so there’s no need for any ‘forklift’ data migration or complex reconfiguration. In one instance, Weta Digital had 10TB of new storage up, provisioned and running in less than three hours, from the time it was delivered.

NetApp stability has never been an issue at Weta Digital, and by clustering the latest FAS3050 filer heads, any single point of failure in the storage architecture has been removed. In the rare instances of disk failure, built in RAID redundancy has protected Weta Digital from data loss and downtime.

Ngan says the whole job has been much easier with Fujitsu. “It’s made my job easier. Product turns up in an astoundingly short period of time. Speed is the critical thing for me, particularly when we are coming to the end of production.”

Digital production on the scale of Weta Digital is always going to thoroughly test the technology and vendor relationships. But both Fujitsu and NetApp have shone through. Weta Digital in 2004 was named NetApp innovator of the year, beating 170 international nominees to claim top honours at a New York ceremony.

FOR MORE INFORMATION//

FUJITSU NEW ZEALAND
+64 9 921 8093
www.fujitsu.co.nz