Google street view is a clever tool which gives us a detailed satellite visual of any place in the world. From the Eiffel Tower to the neighborhood we grew in, google street view has given us a bird’s eye view of places in their exact environments. Engineers at Google have now gone one step further by applying Artificial Intelligence (AI) to turn ordinary satellite picture into images that seem to be taken right out of a photographer’s portfolio.
The AI systems virtual photographer has traveled to areas like the Alps, Banff and Jasper National Parks in Canada, Big Sur in California and Yellowstone National Park, and returned with over 40,000 panoramas, that are quite impressive, judged by professional photographers to be of a professional quality.
Hui Fang, a software engineer on Google’s machine perception team is working on this project using machine learning. It uses a deep learning system for artistic content creation by training a deep neural network to scan thousands of Google street view images in California for shots with an impressive horizon using a generative adversarial network. The software then “mimics the workflow of a professional photographer” fine tuning the pictures into an aesthetically pleasing panorama, something akin to adding filters to enhance the picture quality. Fang then pits the two neural networks against one another and examines the result in order to improve the system.
THE PROCESS OF MAKING ART
To provide aesthetically pleasing photographs, the image is broken down into multiple aspects automatically. AI systems are trained to alter panoramic photos with lighting effects and filters, a task which manually would be time consuming and labor intensive given the requirement of labeled data sets. The system enhances the photos composition, saturation/HDR level with fast and separable optimizations.
A special operation named dramatic mask, which was created jointly while learning the concept of dramatic lighting was also introduced. The Process can be understood below.
The AI system improves upon itself by using another model to distinguish between the edited shot and the original professional image. The end result is software that understands generalized qualities of good and bad photographs, which allows it to then be trained to edit raw images to improve them.
TESTING THE QUALITY
Fang and his team uses a Turing test-like experiment to gauge the efficiency of the system. The images were mixed with other photos at different quality and shown to several professional photographers. They were instructed to assign a quality score for each of them, with meaning defined as follows:
1. Basic: Point-and-shoot without consideration for composition, lighting etc.
2. Average: Good photos from general population without a background in photography. Nothing artistic stands out.
3. Semi-pro: Great photos showing clear artistic aspects. The photographer is on the right track of becoming a professional.
In the following chart, each curve shows scores from professional photographers for images within a certain predicted score range. For our creations with a high predicted score, about 40% ratings they received are at “semi-pro” to “pro” levels.
FUTURE OF GOOGLE STREET VIEW
The Street View panoramas served as a testing bed for the project. Subsequently, Fang and his team, compiled a showcase of photos to this effect which when you see a photo you like, you can click on it to bring out a nearby Street View panorama.
Artificial Intelligence has gone above and beyond to prove that it can perform tasks as well as any human, if not better. Google AI has made some incredible breakthroughs in the field of gaming and job hunting. Continuing along this trend, Google’s AI systems is gearing up for greater challenges that will push its boundaries like never before.