AI/CodeT5
1
0
Fork 0
mirror of https://github.com/salesforce/CodeT5.git synced 2024-10-01 06:35:38 -04:00
Code Issues Packages Projects Releases Wiki Activity

add model predictions

Browse Source
This commit is contained in:
WANG Yue 2023-06-27 12:42:28 +08:00
parent 9fd9e893b7
commit a1215960ef
2 changed files with 32807 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
CodeT5+/README.md
View File

@ -139,6 +139,13 @@ python process_preds.py --path ${output_path} --out_path ${output_path}.jsonl
evaluate_functional_correctness ${output_path}.jsonl
```
We also released the model predictions for our [InstructCodeT5+ 16B](https://huggingface.co/Salesforce/instructcodet5p-16b) at `humaneval/instructcodet5p-16b_T0.2_N200.jsonl` for your reference.
It can reproduce the results of `36.1% Pass@1` with the following command. Note that this result is slightly different from the reported `35.0% Pass@1` in the paper due to the randomness of the sampling process.
```bash
evaluate_functional_correctness humaneval/instructcodet5p-16b_T0.2_N200.jsonl
```
# Citation
```bibtex

32800
CodeT5+/humaneval/instructcodet5p-16b_T0.2_N200.jsonl Normal file
View File

File diff suppressed because one or more lines are too long