This is exactly where the Graz University of Technology (TU Graz) project “RE:Color: Efficient Coloring of Cinema-Quality Films Based on New Machine Learning Methods” started.
However, no real progress was made. The labor and associated costs of manual or semi-automatic staining techniques were too high. In principle, fully automatic coloring of the film is possible. However, while this is a nice looking color, it has the drawback of not corresponding to reality.
Computer scientists led by Thomas Pock at the Institute for Computer Graphics and Vision are working with HS-Art, a Graz-based company that specializes in restoring historic films, to develop interactive and automated coloring techniques and We have developed an integrated software application that combines deep learning. technology.
The result is a largely automated, yet fully user-controlled, algorithm for the staining process.
It is essential that people be able to influence the dyeing process: “There is always a need for people who are familiar with historical traditions and who can explain garments, facades, etc. It looked like it was then. Is a soldier’s uniform green or blue? No algorithm can determine that, but it can learn from it. ”
Therefore, we need to feed the algorithm with a sufficiently large collection of training patterns to automatically take over the coloring of historical films.
“It’s about coloring a movie as efficiently as possible with as little user input as possible. This means, for example, that someone specifies the coloring of his image on the film, and the software decides how to color subsequent frames.” It means taking over,” he explains Pock. The core requirement of this user-guided control is only met by a pre-trained neural network that is dynamically influenced by user interactions.
To this end, researchers have developed various new approaches in the field of automatic coloring based on artificial intelligence (AI). We then worked with the developers at HS-Art to implement the most efficient approach into a prototype application and generated a sufficiently powerful collection of training examples.
We then implemented human-guided controls to get the right color scheme for the real thing. The algorithms that have been developed can be used to restore film very cleanly and in color, but this is not always desirable.