Scaling a Massive State-of-the-art Deep Learning Model in Production

CRANK

commentsBy Lysandre Debut, HuggingFace.Last week, at Hugging Face, we launched a new groundbreaking text editor app. It’s different from traditional text editors in that an NLP model can complete your sentences if you ask it to, bringing a new dimension to “writing with a machine.” It’s based on GPT-2, OpenAI’s language model that can generate syntactically accurate sentences and coherent paragraphs of text.Telling a story with GPT-2’s help.The demo is live on  and you’re welcome to try it out!Write with transformer is to writing what calculators are to calculus.This model is part of the latest trends in NLP, which revolve around creating very large language models that obtain excellent results on a variety of tasks when fine-tuned on those specific tasks. This results in models, “Transformers”, with large amounts of parameters (up to 1.5 billion parameters for GPT-2 Large, or Grover), which are difficult to handle because of their weight.Our app allows the user to choose bet…