Select Page

In-Depth Interview – Jane Lytvynenko

In-Depth Interview – Jane Lytvynenko

We talked to Jane Lytvynenko, senior reporter with Buzzfeed News, focusing on online mis- and disinformation about how big the synthetic media problem actually is. Jane has three practical tips for us on how to detect deepfakes and how to handle disinformation.

Jane Lytvynenko, is a senior reporter with Buzzfeed News, based in Canada. She is primarily focusing on online mis- and disinformation. You can check out her work here. We talked to Jane about how big the synthetic media problem actually is and Jane provides us with practical tips on how to detect deepfakes and how to handle disinformation.

Jane, what is your definition of a deepfake?

Oh, that is a tricky one. I mean it is very different from a video that was just slowed down or manipulated in a very basic way e.g. cut&paste of scenes.

For me a deepfake is using computer technology to make it look like a person said something or took an action they did not say or do.

Where do you mostly encounter deepfakes at the moment?

We do not encounter deepfakes a lot on the day-to-day.  Because the technology is not widely accessible at the moment, we are much more worried about cheapfakes than we are about deepfakes. They spread much faster. We do find deepfakes mostly in satire. We have seen a lot of that come up in the last little while. We also see GAN (Generative Adversarial Networks) generated images being used for fake personas. Essentially faces generated by a computer that are being used to present a persona on social media that doesn’t actually exist in real life.

Do you expect an increase of video synthetic media ahead during the US elections?

No. The reason why I say no is because deepfakes are still fairly difficult to create for people who do not have a lot of tech knowledge. But cheapfakes you can make in iMovie. You can make things using very basic tools that are also convincing.

There is of course always a fear in the back of my head of the ‘Big One’. What is going to be the big deepfake that fools everybody?

Another fear that I have, is a deepfake inserted among legitimate videos. Using one small portion rather than the whole video being a deepfake.

Where do deepfakes have their biggest impact?  

Right now, we see deepfakes mostly used for harassment of women, in pornography in particular. They are primarily targeted at women, but sometimes also at men. Deepfake technology is also being used in movies in Hollywood. And like I mentioned previously for sort of high level production satire.

But when it comes to the field of politics, we’ve seen a couple here and there but in North America we haven’t seen deepfakes that are so convincing that they uproot the political conversation.

Should journalists be concerned?

Do all journalists need to be trained in verification? Or is that a task for experts?

I think it needs to be both. In order to send something to a researcher you need to first understand what it is you are looking for and why you are sending it to a researcher.

So, we need to have basic training. We need reporters to understand what a deepfake looks like. What a GAN-generated image looks like. Get them in on the basics of verification of all types of content. If something extremely technical comes up, they can just send it to the researchers.

For journalists it is very important to have that source. It is very important to have an expert opinion for second verification. That is part of the practice of journalism. But if you do not know to ask the right question, you are not going to be able to get that second opinion.

What are the tools you still miss?

There are a few things. The biggest problem is social media discovery, especially when it comes to video. If there is a video going viral it is fairly difficult to trace back where it came from. There are some tools that break video down into thumbnails. You can then reverse image search them and try to find your way back. But for me right now, there is no way to tell where the video originated. Part of that is a lack of cross-platform searches. For example, Instagram stories; it is one of the most popular Facebook products right now, but they disappear within 24 hours. If somebody downloads that Instagram story and cuts off the banner that says who posted it, they can upload it to Twitter or Facebook, and I will not know where the video came from. It doesn’t allow reporters to see the bigger picture.

Right now, we do not necessarily have the tools to both look at video cross-platform and to look at these videos in terms of when they were shot, what time, by whom, from what angle, at what location. It is a challenge that requires a lot of time that reporters just do not have.

The other thing is, we do not really have a strong way of mapping video spread. So, when reporters do content analysis, they generally focus on text. The reason for that is because text is machine-readable. We have the tools to sort of map out the biggest account that posted this, the smaller accounts that came from it and the sort of audience that looked at this. We do not have similar tools for video even though analyzing the spreading of information is one of the most useful things we do as disinformation reporters. It allows us to see the key points where disinformation traveled. It allows us to understand where to look next time. It allows us most importantly to understand which communities were most impacted.

Is collaboration with researchers and platforms essential in fighting disinformation?

Definitely collaboration with researchers. Platforms are a bit on and off again in terms of what kind of information they are willing to provide us with. Sometimes they are willing to confirm fact findings, but they are rarely helping us to do research independently.

This is where researchers, analytics, sort of third parties that specialize in this are really key for reporters.

How should we report about disinformation and deepfakes?

At Buzzfeed News we always try to put correct information first. We repeat the accurate information before you get to the inaccurate information. There are two different approaches you can take. One is reporting on the content of the video and the other is reporting on the existence of the video as well as any information you have in terms of where it came from, who posted it and why.

We generally focus on the second approach as the primary presentation of facts.

That is how we frame a lot of these things. Because the key aim of a manipulated video is to get the message across and if we put the message at the top then they still get the message across. What you want to do is describe the techniques, describe how they are attempting to manipulate the audience. And then explain the other part of manipulation which is the message.

Do you have a specific workflow in verifying digital content?

We do have best practices. Putting accurate information first is definitely the top priority. We also make sure to never put up an image or a screenshot without stamping it or crossing it out in some way. That gives a visual clue to anybody who comes across it that it is false. But more importantly, if a search engine scrapes that image and somebody comes across it on Google or Bing they are immediately able to see that it is false.

Buzzfeed false stamp

In terms of workflow for verification, the key part is documentation. We archive everything that we come across and take a screenshot. We are essentially making sure that we are able to retrace our steps. It is kind of a scientific process, we want to make sure that if anybody repeated our steps, they would come to the same conclusion. A lot of the times when we do pure debunks that is what we focus on. Because not only does it increase trust and shows how we got to the conclusion, it also teaches our audience some of the techniques that we are using so that they can use them in the future as well.

Can you still trust what you are seeing, or do you always have this critical view?

The short answer is yes, especially with a lot of videos. I really fear missing something because sometimes the manipulation is so subtle you can’t quite tell that it is a manipulation from the first or the second look. If somebody is just scrolling on their feed, they are not looking very closely at those details. They might not even listen to the audio and hear that it sounds off. They might read the subtitles instead. They might not notice that the mouth in a deepfake is a little bit imperfect because those are little details. We are bombarded with information in our news feed so we might just not notice it.

I’m always extremely suspicious and sometimes I’m more suspicious than I should be, sometimes I look at a video and I’m like was that slowed down by 0.7 of a second or am I losing my mind?

What are the three main tips for a news consumer to detect synthetic media?

Tip #1
My first tip, if you see a video that is extremely viral or if you see video that sparks a lot of emotion or if you see a video that just kind of like feels a little bit off
just pause.

From there the number one thing you can do is search a couple of key terms in a search engine with the words fact check to see if somebody has already picked up on what you are seeing. You can also read the comments, very often in the comments people will explain what is going on in the video.

Tip #2
If you are unsure about the video really do play the audio and look at the key features that make people people. Look at the eyes, look at the mouth. What are the mannerisms of the person that you are seeing in the video and do they match with what you understand about that person? Ask yourself if the voice of the person sounds like their real voice.

A lot of people when they see a GAN-generated photo for example, they have a gut feeling that something is wrong. They feel that they are not looking at a real person, but they can’t quite explain why. So, just really trust that feeling and start looking for those little signs that something is wrong. If it is a photo usually the best thing to look at are the earlobes. If a person has glasses look at the glasses. Eyebrows are not generally perfect if a photo is computer-generated and teeth are always a little off. Those are the things that I would look for.

Tip #3

The final tip is: do not share anything you are not sure of. Do not pass it on to your network.

We all have created a small online community around us, whether it is friends, family, acquaintances or sort of strangers that we met on the internet. And most of that community really trusts us. Even if you are not a public figure your friends are going to trust what you post. So, take that responsibility seriously and try to not pass on anything that you are unsure of to that online community.

When not deepfakes, what else would be our challenge in disinformation?

My biggest worry when it comes to disinformation is not necessarily synthetic media. It is humans trying to convince other humans.

Look at the most insidious falsehoods that we see right now in the US; the QAnon mass delusion like we call it at Buzzfeed. People who believe in this are very often brought on board by other people they know. So, what I really worry about is the continuing creation of online communities where people bring one another along for the ride except the ride is extremely false.

I think that manipulated images and manipulated videos and fake news articles are all just tools. They are all parts of the problem. But the problem itself I think is a community problem and I definitely foresee that community problem growing beyond synthetic media.

Many thanks Jane! If you are interested to learn more or have questions then please get into contact with us, either via commenting on this article or via our Twitter channel.

We hope you liked it! Happy Digging and keep an eye on our website for future updates!

 Don’t forget: be active and responsible in your community – and stay healthy!

Related Content

From Rocket-Science to Journalism

From Rocket-Science to Journalism

In the Digger project we aim to implement scientific audio forensic functionalities in journalistic tools to detect both shallow- and deepfakes. At the Truth and Trust Online Conference 2020 we explained how we are doing this.

Audio Synthesis, what’s next? – Mellotron

Audio Synthesis, what’s next? – Mellotron

Expressive voice synthesis with rhythm and pitch transfer. Mellotron managed to let a person sing, without ever recording his/her voice performing any song. Interested? Here is more…

Video verification step by step

Video verification step by step

What should you do if you encounter a suspicious video online? Although there is no golden rule for video verification and each case may present its own particularities, the following steps are a good way to start.

From Rocket-Science to Journalism

From Rocket-Science to Journalism

In the Digger project we aim to implement scientific audio forensic functionalities in journalistic tools to detect both shallow- and deepfakes. At the Truth and Trust Online Conference 2020 we explained how we are doing this.

Today we have presented the Digger project and our approach on the Truth & Trust Online Conference. We explained all the audio forensic functionality that we are looking into and how this functionality can help us with detecting video manipulation and also with deepfake detection. The presentation is available via the conference website but we did not want to keep it from you so below you will find the full 11 minutes presentation.

If you are interested to learn more or have questions then please get into contact with us, either via commenting on this article or via our Twitter channel.

We hope you liked it! Happy Digging and keep an eye on our website for future updates!

 Don’t forget: be active and responsible in your community – and stay healthy!

Related Content

From Rocket-Science to Journalism

From Rocket-Science to Journalism

In the Digger project we aim to implement scientific audio forensic functionalities in journalistic tools to detect both shallow- and deepfakes. At the Truth and Trust Online Conference 2020 we explained how we are doing this.

Audio Synthesis, what’s next? – Mellotron

Audio Synthesis, what’s next? – Mellotron

Expressive voice synthesis with rhythm and pitch transfer. Mellotron managed to let a person sing, without ever recording his/her voice performing any song. Interested? Here is more…

Video verification step by step

Video verification step by step

What should you do if you encounter a suspicious video online? Although there is no golden rule for video verification and each case may present its own particularities, the following steps are a good way to start.

Video verification step by step

Video verification step by step

What should you do if you encounter a suspicious video online? Although there is no golden rule for video verification and each case may present its own particularities, the following steps are a good way to start.

What should you do if you encounter a suspicious video online? Although there is no golden rule for video verification and each case may present its own particularities, the following steps are a good way to start. 

Pay attention and ask yourself these basic questions

Start with asking some basic questions like “Could what I am seeing here be true?”, “Who is the source of the video and why am I seeing/receiving this?”. “Am I familiar with this account?”, “Has the account’s content and reporting been reliable in the past?” and “Where is the uploader based, judging by the account’s history?”. Thinking the answers to such questions may raise some red flags about why you should be skeptical towards what you see. Also, watch the video at least twice and pay close attention to the details; this remains your best shot for identifying fake videos, especially deepfakes. So, careful viewers may be able to detect certain inconsistencies in the video (e.g. non-synchronized lips or irregular background noises) or signs of editing/manipulation (e.g. certain areas of a face that are blurry or strange cuts in the video). Most video manipulation is still visible by the naked eye. If you want to read more on how to deal with dubious claims in general, you can read our previous blog post

Capture and reverse search video frames

When encountering a suspicious image, reverse searching it on Google or Yandex is one of the first steps you take in order to find out if it was used before in another context . For videos, although reverse video search tools are not commercially available yet, there are ways to work around that, in order to examine the provenance of a video and see whether similar or identical videos have circulated online in the past. There are many tools like Frame-By-Frame that enable users to view a video frame-by-frame, capture any frame and save it – if you have the VLC player installed it works as well. 

Cropping certain parts of a frame or flipping the frame (flipping images is one method disinformation actors use to make it more difficult to find the original source through reverse image search) before doing a reverse search may sometimes yield unexpected results. Also, searching in several reverse search engines (Google, Yandex, Baidu, TinEye, Karma Decay for Reddit, etc.) increases the possibility of finding the original video. The InVID-WeVerify plugin can help you verify images and videos using a set of tools like contextual clues, image forensics, reverse image search, keyframe extraction and more.

Examine the location where the video was allegedly filmed

Although in some instances it is very difficult or nearly impossible to verify the location where a video was shot, other times the existence of landmarks, reference points or other distinct signs in the video may reveal its filming location. For example, road signs, shop signs, landmarks like mountains, distinct buildings or other building structures can help you corroborate the video’s filming location.

Tools like Google Maps, Google Street View, Wikimapia, and Mapillary can be used to cross-check whether the actual filming location is the same as the alleged. Checking historical weather conditions for this particular place, date and time is another way to verify a video. Shadows visible in the video should also be cross-checked to determine whether they are consistent with the sun’s trajectory and position at that particular day and time. SunCalc is a tool that helps users check if shadows are correct by showing sun movement and sunlight phases during the given day and time at the given location. And sometimes it helps to stitch together several keyframes to narrow down the location – you may check this great tutorial by Amnesty

Video metadata and image forensics 

Even though most social media platforms remove content metadata once someone uploads a video or an image, if you have the source video, you can use your computer’s native file browser or tools like Exiftool to examine the video’s metadata. Also, with tools like Amnesty International’s YouTube DataViewer you will be able to find out the exact day and time a video was uploaded on YouTube.  If the above steps don’t yield confident results and you are still unsure of the video you can try out some more elaborate ways to assess its authenticity. With tools like the InVID-WeVerify plugin or FotoForensics you can examine an image or a video frame for manipulations with forensics algorithms like Error Level Analysis (ELA) and Double Quantization (DQ). The algorithms may reveal signs of manipulation, like editing, cropping, splicing or drawing. Nevertheless, to be able to understand the results and draw safe conclusions avoiding false-positives a level of familiarity with image forensics is required.

A critical mind and an eye for detail

As mentioned above, there is no golden rule on how to verify videos. The above steps are merely exhaustive, but they can be a good start. But as new methods of detection are developed, so are new manipulation methods – in a game that doesn’t seem to end. The commercialization of the technology behind deepfakes through openly accessible applications like Zao or Doublicat is making matters worse driving the “democratization of propaganda”. What remains most important and independent of the tools that can be used for the detection of manipulated media is to approach any kind of online information (especially user generated content) with a critical mind and an eye for detail. Traditional steps in the verification process, such as checking the source and triangulating all available information still remain central.   

In the effort to tackle mis- and disinformation, collaboration is key. In Digger we work with Truly Media to provide journalists with a working environment where they can collaboratively verify online content. Truly Media is a collaborative platform developed by Athens Technology Center and Deutsche Welle that helps teams of users collect and organise content relevant to an investigation they are carrying out and together decide on how trustworthy the information they have found is.  In order to make the verification process as easy as possible for journalists, Truly Media integrates a lot of the tools and processes mentioned above, while offering a set of image and video tools that aid users in the verification of multimedia content. Truly Media is a commercial platform – for a demo go here.

How to get started?

If you are a beginner in verification or if you would like to learn more about the whole verification process, we would suggest reading the first edition of the Verification Handbook, the Verification Handbook for Investigative Reporting, as well as the latest edition published in April 2020.

Stay tuned and get involved

We will publish regular updates about our technology, external developments and interview experts to learn about ethical, legal and hand-on expertise.

The Digger project is developing a community to share knowledge and initiate collaboration in the field of synthetic media detection. Interested? Follow us on Twitter @Digger_project and send us a DM or leave a comment below.

Related Content

In-Depth Interview – Jane Lytvynenko

In-Depth Interview – Jane Lytvynenko

We talked to Jane Lytvynenko, senior reporter with Buzzfeed News, focusing on online mis- and disinformation about how big the synthetic media problem actually is. Jane has three practical tips for us on how to detect deepfakes and how to handle disinformation.

Audio Synthesis, what’s next? – Mellotron

Audio Synthesis, what’s next? – Mellotron

Expressive voice synthesis with rhythm and pitch transfer. Mellotron managed to let a person sing, without ever recording his/her voice performing any song. Interested? Here is more…

Video verification step by step

Video verification step by step

What should you do if you encounter a suspicious video online? Although there is no golden rule for video verification and each case may present its own particularities, the following steps are a good way to start.