The fitting rooms of the future: CLOTH3D

A team of the University of Barcelona and the Computer Vision Center (CVC) have carried out CLOTH3D, the first 3D synthetic dataset at a large scale aimed at the development of deep learning to simulate clothes on different body types. This dataset, created artificially and published on open access, is the first step to improve virtual fitting rooms.

There are more and more people who buy clothes online every day. But there are some disadvantages. One of the most important issues is that people cannot try clothes on before receiving them. To solve this problem, 3D clothes generation and modelling have been used, a key for artificial intelligence and deep learning. These models, which will ease the work of designers and image animators, involve an improvement in the experience the virtual fitting rooms provide.

Nowadays, there are models to simulate clothes on different bodies, but many are in 2D. The reason is that 3D models need many data, and those that are currently available are just a few. There are three main strategies to produce 3D data for dressed people: 3D scanning, 3D image generation with conventional images and synthetic generation.

• 3D scanning is expensive and unable to differentiate clothes from body.
• On the other hand, datasets that infer 3D clothing geometry with conventional images are inexact and cannot model properly the dynamics of the clothes.
• However, synthetic data are easy to create and are free from measurement errors.

A team from the Human Pose Recovery and Behaviour Analysis Group, from the Computer Vision Center (CVC) and the University of Barcelona (UB), chose the latter path and created CLOTH3D, the first set of synthetic data at a large scale with 3D human dressed sequences, published recently in the journal Computer Vision – ECCV 2020 Workshops. “Since we need many data to create 3D models, we decided to create our own data. We designed and published the largest dataset of this kind with a great range of clothes and movements”, notes Hugo Bertiche (UB – CVC) also member of the Institute of Mathematics of the UB (IMUB).

With more than two million samples, CLOTH3D is unique when it comes to variability, regarding the type of clothes, shape, size, tightness and tissue. The fitting room can simulate thousands of poses and different body types, which creates a real dynamic for clothes.

However, there is a lot of work to do still: Real clothes are more complex than what simulators can show, so deep learning has to find the right way to model extremely fine and chaotic details, such as wrinkles, and arbitrary geometrical objects related to dresses, such as hats, glasses, gloves, shoes and ornaments.

Source: Universidad de Barcelona