Posted in Information Technology

Kristen Stewart co-authored a paper on style transfer and the AI community lost its mind

https://techcrunch.com/2017/01/19/kristen-stewart-co-authored-a-paper-on-style-transfer-and-the-ai-community-lost-its-mind/?ncid=rss&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+Techcrunch+%28TechCrunch%29&sr_share=facebook

Hollywood and the academic artificial intelligence community have an interesting relationship, to say the least. Many people interested in the field, myself included, love science fiction, but regularly end up being the obnoxious commentator on family movie night pointing out that Johnny Depp’s Transcendence is more anti-intelligence than artificial intelligence. So it wasn’t all that surprising that researchers were a bit perplexed when actress and director Kristen Stewart appeared as an author on a machine learning paper.

François Chollet

@fchollet

To be someone in Hollywood, you’ve got to put your ML papers on Arxiv and you better use TensorFlow… https://twitter.com/petewarden/status/822154283315433472 

Pete Warden@petewarden

Kristen Stewart cites @tensorflow in a research paper about style transfer in her latest movie! https://arxiv.org/abs/1701.04928 

138 people are talking about this

Mark Riedl 🚀 Mars (Moon)@mark_riedl

Once more: Kristen Stewart of Twilight fame directs movie; writes arXiv paper about using StyleNet during production https://arxiv.org/abs/1701.04928 

Mark Riedl 🚀 Mars (Moon)@mark_riedl

I used to tell Twilight jokes in my Intro to AI class. I take it all back.

16 people are talking about this

Mark Riedl 🚀 Mars (Moon)@mark_riedl

Once more: Kristen Stewart of Twilight fame directs movie; writes arXiv paper about using StyleNet during production https://arxiv.org/abs/1701.04928 

Mark Riedl 🚀 Mars (Moon)@mark_riedl

I used to tell Twilight jokes in my Intro to AI class. I take it all back.

16 people are talking about this

Stewart directed Come Swim, a short film about a man’s day told through both impressionism and realism. The film features the use of the so-called “style transfer” technique to build its story.

Prisma Facebook Live video testStyle transfer is the fun technique that involves using convolutional neural networks to artistically alter video in real time. To do this, you basically show your algorithm an image (say a painting by van Gogh) and then teach it to apply the same painting technique to any regular photo.

The short, three-page paper, entitled Bringing Impressionism to Life with Neural Style Transfer in Come Swim, is really just a high-level case study about what it’s like to apply style-transfer techniques to cinema.

In addition to Stewart, Bhautik Joshi and David Shapiro of Adobe and Starlight Studios, respectively, speak to the challenges of trying to reign in style transfer for controlled use. The team’s approach comes off a bit like blindfolded dart-throwing, but, to be fair, the trial and error process and dealing with trade-offs is pretty much what machine learning research is like at all levels.

One would almost expect that artists and movie-makers would be more comfortable with improvisation than researchers, but this doesn’t seem to be the case. “In a production setting, however, a great deal of creative control is needed to tune the result, and a rigid set of algorithmic constraints run counter to the need for this creative exploration,” note the authors of the paper.

This isn’t the first time that style transfer has been used for artistically expressive purposes. Some in the research community were quick to point out that a paper was written recently about the movie Interstellar. And on the other side of the table, style-transfer techniques have been used for lower-profile artistic videos in the past.

The prospect is none-the-less interesting and it’s likely we will see more of style transfer and machine learning in general in film making. Machines have already produced musicmovie trailers and scripts. So while it seems like humans still remain on top, just ask yourself, how long is it until machines can create something better than [insert your favorite bad movie here].

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s