We had an initial bug in logging the training loss but we noticed a decrease in validation loss.
During generation, we noticed that our model seemed to never generate an `eos` token. We realized that this was due to the fact that we set
the tokenizer `pad` token equal to the `eos` token and most likely never learned to "stop". However, generations seemed to be working
well, even with greedy generation.
For example, when prompted with `Generate a python script to reverse a string.`, the model outputs valid markdown and a working script
Here's a Python script to reverse a string:
```python
def reverse_string(string):
return ''.join(reversed(string))
```
This function takes a string as an argument and returns the reversed string. The `reversed()` function reverses the string and returns a list of characters. The `join()` function joins the list of characters into a string.
You can use this function like this:
```python
string = 'hello world'
print(reverse_string(string))
```
This will output `'olleh dna hwl'`.
Note that this function does not handle strings with spaces or other special characters. If you need to handle those cases, you can use a regular expression to split the string into words and then reverse each word.
```python
import re
def reverse_string(string):
words = re.split(' ', string)
return ''.join(reversed(word) for word in words)
```
This function takes a string as an argument and returns the reversed string. The `re.split()` function splits the string into words using a space as the delimiter. The `reversed()` function reverses each word and returns a list of characters. The `join()` function joins the list of characters into a string.
You can use this function like this:
```python
string = 'hello world'
print(reverse_string(string))
```
This will output `'olleh dna hwl'`.
Note that this function also handles strings with spaces or other special characters.
Answer: Here's a Python script to reverse a string:
```python
def reverse_string(string):
return ''.join(reversed(string))
```
This function takes a string as an argument and returns the reversed string. The `reversed()` function reverses the string and returns a list of characters. The `join()` function joins the list of characters into a string.
You can use this function like this:
```python
string = 'hello world'
print(reverse_string(string))
```
This will output `'olleh dna hwl'`.
Note that this function does not handle strings with
although it duplicates and continues generating past answering the full question.
The base model doesn't answer the question and hallucinates
```
Write a python script to reverse a string. The string should be reversed in place.
The string should be reversed in place.
The string should be reversed in place. The string should be reversed in place.
The string should be reversed in place. The string should be reversed in place. The string should be reversed in place.
The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place.
The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place.
The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place.
The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place.
The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place.
The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place.
The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place.
The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be reversed in place. The string should be
```
## EOS and Accidental Duplication
Seeing as the model never stopped, we experimented with training a model with a separate token for `eos` and `pad`. Since we couldn't find a `pad` token present in the vocabulary, we added one to the tokenizer and expanded the embedding size of the model (from 32,000). In theory, we could have expanded the embedding size by a factor of 64 to improve throughput and performance, as [noted here](https://twitter.com/ctnzr/status/1623758178587648000?s=20).
For every sequence, we appended an `eos` token in hopes the model would learn to exit.
We successfully trained a model using the same parameters as before 
During generation, our model exited early even with greedy generations
You can use the `reversed()` function to reverse a string in Python. Here's an example:
```python
string = 'hello world'
reversed_string = reversed(string)
print(reversed_string)
```
Output:
```
world hello
```
For a harder prompt where we try to trick the model with comments
```python
#this code prints a string reversed
my_string = "hello how are you"
print(len(my_string))
My code above does not work. Can you help me?
```
The model correctly answers
The code you provided does not print a string reversed. It prints the length of the string "hello how are you".
We realized that we had two bugs however:
- We accidentally duplicated data and effectively trained for 2 epochs instead of 1
- We added an eos token to every sequence, even those that we truncated (e.g. long code that exceeds the 1024).
Using the same parameters, we then trained a model using a "conditional" eos token where we only add an `eos` when the inputs are less than the maximum sequence length for one epoch.
Our training and validation loss looks fairly normal
However, we found generations to be slightly poorer. For the same prompt of `Generate a python script to reverse a string.`, the model generates
Write a python script to reverse a string.
Here is a python script to reverse a string:
import string
def reverse_string(string):
return string[::-1]
print(reverse_string('hello world'))
Output:
world hello
The prompt
```python
#this code prints a string reversed
my_string = "hello how are you"
print(len(my_string))
My code above does not work. Can you help me?
```
does not generate any text.
And the prompt `"Generate a python script to make a get request to an api endpoint."` generates
I'm sorry, I cannot provide a specific answer to this question as it requires more context and details about the API endpoint and the specific task you are trying to accomplish. Can you please provide more information?
## Multi Epoch and Full Model Training
We decided to remove the entire Bigscience/P3 subset from the final training dataset due to data diversity considerations.
P3 contains many homogeneous prompts which produce short and homogeneous responses from GPT-3.5-Turbo.
Taking inspiration from [the Alpaca Repo](https://github.com/tatsu-lab/stanford_alpaca), we roughly scale the learning rate by `sqrt(k)`, where `k` is the increase in batch size, where Alpaca used a batch size of 128 and learning rate of 2e-5.
Comparing our model LoRa to the [Alpaca LoRa](https://huggingface.co/tloen/alpaca-lora-7b), our model has lower perplexity. Qualitatively, training on 3 epochs performed the best on perplexity as well as qualitative examples.
We tried training a full model using the parameters above, but found that during the second epoch the model diverged and samples generated post training were worse than the first epoch.
We trained multiple [GPT-J models](https://huggingface.co/EleutherAI/gpt-j-6b) with varying success. We found that training the full model lead to diverged post epoch 1. 
Using Atlas, we extracted the embeddings of each point in the dataset and calculated the loss per sequence. We then uploaded [this to Atlas](https://atlas.nomic.ai/map/gpt4all-j-post-epoch-1-embeddings) and noticed that the higher loss items seem to cluster. On further inspection, the highest density clusters seemed to be of prompt/response pairs that asked for creative-like generations such as `Generate a story about ...` 
We varied learning rate, learning rate schedule, and weight decay following suggestions from the [original GPT-J codebase](https://github.com/kingoflolz/mesh-transformer-jax/blob/master/howto_finetune.md) but found no real performance difference (qualitatively or quantitatively) when varying these parameters.
The final model was trained using the following hyperparameters with a linear warmup followed by constant learning rate:
| Hyperparameter | Value |
|----------------|-------|
| Per Device BS | 32 |
| Global BS | 256 |
| Learning rate | 2e-5 |
| Epochs | 2 |
| Max length | 1024 |
| Weight decay | 0 |
| Warmup Steps | 500 |
The LoRA model was trained using using the following hyperparameters with a linear warmup followed by constant learning rate:
