Citation numbers should be after the full stop instead of before

[bryceerobertson]

See e.g. Wikipedia.

[mruwnik]

requires prompt engineering, as this is returned by the LLM

[bryceerobertson]

@melissasamworth who's in charge of prompt engineering?

[LeMurphant]

@bryceerobertson afaik there is not a single person, but Matthew (@matthew1970 on Discord) has been poking around with prompt engineering. AFAIK he is not on Github

[mwatkins1970]

I've submitted a PR (#148) that fixes this issue by updating the example in settings.py's QUESTION_PROMPT to show the period before the citation brackets. This should help ensure consistent citation formatting throughout responses.

[LeMurphant]

I'm not sure it's possible to cross-link between projects, but here's a try
#148
Edit: It works!

[bryceerobertson]

Update: @LeMurphant will give this a go using the following recommendation from Matthew:

Most frustratingly, I can't run the code on my local machine (outdated MacOS won't support all the necessary packages), so I can't test it.

As I wrote in the github pull request notes:

"This was simply a matter of changing one line in stampy-chat/api/src/stampy_chat/settings.py

QUESTION_PROMPT = (
"In your answer, please cite any claims you make back to each source "
"using the format: [1], [2], etc. If you use multiple sources to make a claim "
"cite all of them. For example: "AGI is concerning [1, 3, 8]."\n\n"
)
to

QUESTION_PROMPT = (
"In your answer, please cite any claims you make back to each source "
"using the format: [1], [2], etc. If you use multiple sources to make a claim "
"cite all of them. For example: "AGI is concerning. [1, 3, 8]" (note the period placement)\n\n"
)

I am unable to run this codebase on my local machine (due to dependencies issues caused by an outdated MacOS, I think), but I'm 98% confident this will work, since the reason for the unwanted behaviour was so explicit."
So perhaps rather than me iterating over variations of QUESTION_PROMPT via github, someone could with a better setup could just try a few variations. I would suggest changing

For example: "AGI is concerning. [1, 3, 8]" (note the period placement)\n\n"

to

For example: "AGI is concerning. [1, 3, 8]" (note that the period is placed before the citation numbers)\n\n"

or even

For example: "AGI is concerning. [1, 3, 8]" (it is very important that the period is placed before the citation numbers)\n\n"

I used the most terse wording I could to save on token expenditure, but I'm confident that a more explicit instruction would do the job.

[LeMurphant]

I found it challenging to setup my stampy-chat environment, @jrhender as discussed can you see if you can do it?