Fine-tuning a language model to create an assistant
My goal is to fine-tune a model by training it on synthetic company data in the form of text reports, i.e. monthly/weekly reports about the company's performance
I then want to be able to prompt the model with questions such as "What are the company's biggest weaknesses?" or "What should the company's strategy be for next month?". So I want the fine-tuned model to be able to act as an advisor to the company given its understanding of the company's reports/performance
Given that my company report data is a load of text files, what is the best way to structure it for my use case? Can I simply train the model on a multiple text strings, should I convert it into question-answer pairs, a combination of strings and pairs, or do you suggest another approach?