NLP

Natural Language Processing (NLP) is using machine learning techniques to work with text. Natural language is a language that has developed naturally in use (as contrasted with an artificial language or computer code).

Transfer learning

• The default learning rate for Adam is usually way too high for transfer learning, one thing you can do is have a decaying learning rate for the optimizer. For example, you can use a learning rate based on a polynomial learning rate.

HuggingFace Course on fine-tuning a pre-trained model

Links

• Introduction Course on NLP from Hugging Face
• Glossary from HuggingFace
• HuggingFace Accelerate Examples Repo

Thoughts