A.I Generated text detection application can be built upon E2E Cloud using GLTR(Giant Language Model Test Room)
In this post we will discover :
- About GLTR
- User Interface for GLTR
- Backend of GLTR
- Empirical Validation and its findings
- Qualitative study of human-subject study
- Using the GLTR
- Extending the Backend of GLTR for a new model
- Extending the frontend
Let's look at the few high profile examples of poor use of text generation:
- A simple generational system was used to create fake comments in opposition to net neutrality(Grimaldi, 2018).
- Generation of false articles (Wang, 2017)
- Misleading reviews (Fornaciary and Poesio, 2014)
This is evident that a forensic technique is necessary to detect this automatically generated text. Also the technique should be easy enough to convey to amateur and requires minimal setup cost. One such tool that we are going to discover in this post is GLTR(Giant Language Model Test Room) which is based on visualization and is easy to interpret.
Assumption:- The system over generates high confidence text from a limited subset of true distribution of natural language.
Experimentation & Result:- The result of various empirical studies (References) led to the conclusion that real text uses a wider subset of the distribution under a model. This is visible when most distributions are low entropy & concentrate probability in limited words.
Image Source:arXiv:1906.04043 [cs.CL]
Without the tool the success rate of differentiation between real and generated text was 52%, which is very close to a random guess but with the help of a tool,without any prior training, it is calculated 72%.
User Interface for GLTR :
Image Source:arXiv:1906.04043 [cs.CL]
On the top, we have shown three graphs with global information:
(a). Below the graphs, users graphs ,user can switch between two different annotationsannotation s and customize the top-k thresholds.
(b). On the bottom,each token is shown with the associated annotation as heatmap.
(c). The tooltip (d) highlights information about the current prediction, when hovering over the word “chuck”.
Backend study of GLTR: Let's dive into the concepts used to make this tool.
Task: To detect whether a sequence of words X1:N is written by a human or A.I generatedthree tests are performed using a detection model (We assume access to a different model of same form) to access whether text is generated in the following way:-
- Test 1:- The probability of the word, e.g. pdet(Xi = Xˆ i |X1:i−1), [ To check whether a generated word is samples from the top of the distribution (rRecall the assumption)].
- Test 2:- The absolute rank of a word e.g. rank in prediction pdet(Xi = Xˆ i |X1:i−1),
[ To check whether a generated word is sampledsamples from the top of the distribution (rRecall the assumption)].
- Test 3:-The entropy of the predicted distribution e.g. − ∑wpdet(Xi = w|X1:i−1) log pdet(Xi = w|X1:i−1). ([To find whether the previously generated context is well known to the detection system such that it is overly sure of its next prediction).
Implementation of Tests:
The publicly deployed version (http://gltr.io/) uses both BERT (Devlin et al., 2018) and GPT-2 (Radford et al., 2019) 117M .
For GPT-2 we compute pdet(Xi | X1…..i−1) at each position i in a text X since it is a left to right language model.
For BERT we iteratively masked out each correct token Xi and use a context of 30 words on each side of each input to estimate pdet(Xi |Xi−30...i−1, Xi+1...Xi+30)2 . By default, a word that ranks within the top 10 is highlighted in green, top 100 in yellow, top 1,000 in red, and the rest in purple.
The backend of GLTR is implemented in PyTorch and is designed to ensure extensibility. New detection models can be added by registering themselves with the API and providing a model and a tokenizer. This setup will allow the frontend of the tool to continue to be used as improved language models are released.
Case Study:
Image Source:arXiv:1906.04043 [cs.CL]
The analysis shows that not a single token in the generated text is highlighted in purple and only a few in red. Most words are green or yellow, indicating high rank. Additionally, the second histogram shows a high fraction of high probability choices. A final indicator is the regularity in the third histogram with a high fraction of low-entropy predictions and an almost linear increase in the frequency of high-entropy words.
In contrast, we show two human-written samples; one from a New York Times article and a scientific abstract (Figure 3d+e). There is a significantly higher fraction of red and purple (e.g. nonobvious) predictions compared to the generated example. The difference is observable in the histograms that the fraction of low-probability words is higher and low-entropy contexts are comparatively smaller.
Empirical Validation:
50 articles from 3 generated resources and 3 human resources are taken.
Generated Resources :-
- GPT-2 1.5 B (Radford et al., 2019) random subset of released examples with temperature 0.7
- GPT-2 1.5 B (Radford et al., 2019) random subset of released examples truncated to top 40 predictions
- Washington Post Heliograf system which cover local sports results
Human Resources:-
- New York Times articles (NYT)
- Scientific abstracts from the journals nature and science (SA)
The samples are taken only from post release of GPT-2.
Findings of Empirical Validation:
ImageSource:arXiv:1906.04043 [cs.CL]
Real texts use words outside of the top 100 predictions 2.41 times as frequently under GPT-2 (1.67 for BERT) as generated text, even compared to sampling with a lower temperature.
We hypothesize that human authors use low-rank words, even when the entropy is low, a property that sampling methods for generated text avoid . As shown in Figure 5, human text uses high-rank words more frequently, regardless of the estimated entropy.
ImageSource:arXiv:1906.04043 [cs.CL]
As shown in Figure 5, human text uses high-rank words more frequently, regardless of the estimated entropy.
Qualitative findings of a Human-Subject Study:-
A study with 35 volunteer students in college level NLP class was done to distinguish real text with generated text.The tool caused students to think about the properties of the fake text. While humans would vary expressions in real texts, models rarely generate synonyms or referring expressions for entities, which does not follow the theory of centering in discourse analysis (Grosz et al., 1995).
Using the GLTR on E2E Cloud:-
Since GLTR is an open source yet influential tool, you can use it on E2E Cloud. We assume a background of launching cloud instances and utilizing github repositories.
Link for github repo:-
https://github.com/HendrikStrobelt/detecting-fake-text.git
Quickstart:-
Install dependencies for Python >3.6 :
pip install -r requirements.txt
run server for gpt-2-small:
python server.py
Start the server for BERT:
python server.py --model BERT
Extend the backend on E2E Cloud:-
The GLTR is not limited up to a certain model and can be extended on new models. E2E platform provides you with an affordable cloud GPU you can utilize to extend GLTR for your application.
The backend defines a number of model api's that can be invoked by the server by starting it with the parameter --model NAME. To add a custom model, you need to write your own api in backend/api.py and add the decorator @register_api(name=NAME).
Each api needs to be a class that inherits from AbstractLanguageChecker, which defines two functions check_probabilities and postprocess.
Please follow the documentation within api.py (refer the github repo https://github.com/HendrikStrobelt/detecting-fake-text.git)
when implementing the class and the functions.
Extended the Frontend on E2E Cloud:-
The source code for the front-end is in client/src. (https://github.com/HendrikStrobelt/detecting-fake-text.git )
To modify, installing of node dependencies is necessary:
cd client/src; npm install; cd ../..
re-compilation of front-end:
> rm -rf client/dist;cd client/src/; npm run build; cd ../..
E2E cloud provides theprovide best GPUbest of GPU which can elevate your application significantly at an affordableat affordable price.
Contact us for your free trial: sales@e2enetworks.com
References:-
Tommaso Fornaciari and Massimo Poesio. 2014. Identifying fake amazon reviews as learning from crowds. In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, pages 279–287.
William Yang Wang. 2017. ” liar, liar pants on fire”: A new benchmark dataset for fake news detection. arXiv preprint arXiv:1705.00648.
James V. Grimaldi. 2018. U.s. investigating fake comments on net neutrality. The Wall Street Journal.
arXiv:1906.04043 [cs.CL]
