[DTOSS-11551] Add extract model

[cameronhargreaves1-nhs]

Description

we want to model the relationship between Appontments and the Extract it was booked in properly to give us the ability to more easily check sequence numbers.

Whenever appointments are created, a new Extract will need to be created as well.

Jira link

Review notes

Review checklist

• Check database queries are correctly scoped to current_provider

[Harriethw]

Looks pretty good so far! just some minor changes/ questions

[steventux]

Looking good, a couple of small things, the main thing is around how we read the header, I think we can make it more readable.

[steventux]

There are a lot of magic indexes to what we are doing here, it makes the code hard to read when we don't know what file_headers[1] or file_headers[4] refer to.
Could we have a pattern like:

def create_extract(filename: str) -> Extract:
    bso_code = filename.split("_")[0]
    type_id, extract_id, start_date, start_time, record_count = raw_data.split("\r\n").split("|")
    # Maybe raise if the above fails
    return Extract.objects.create(
          sequence_number: extract_id,
          bso_code: bso_code,
          filename: filename,
          record_count: record_count,
    )

[Harriethw]

just seen this doesn't look like it was added, will take a look

[Harriethw]

@steventux have pushed in latest commit - IMO the string manipulation is harder to understand than just converting to a dataframe, what do you think?

[Harriethw]

there were a few more steps to parsing the string than in your original suggestion - i guess we get all that "for free" from the pandas stuff

addressed changes

[Harriethw]

I tidied up the commits to make it easier to grep, but there are a few changes of note from the original PR:

• adding the related_name on the ManyToMany allows us to query extracts from the Appointment, which I think is useful to have
• the validation on a unique constraint (e.g. if we try to process same Extract file twice) does happen on create once the Model is setup right - so no new Extract would be created, and we would be alerted.
• I had to pass transaction=True into the unique test because pytest was struggling and didn't seem to the think the transaction was atomic - I think it should be because we've wrapped everything in a try/catch block, but not 100% sure 🤔

[steventux]

This looks great, very clean and well tested 💯 🥇 🛳️

[steventux]

I don't know if this matters or even accurate but the spec states that the line separator is CR/LF which is \r\n in our money. I suppose the only side effect of splitting on \n would be rogue carriage returns.
Perhaps as we are already stripping quotes we could attempt to strip \r?
Not a dealbreaker.

[Harriethw]

done!
I didn't know how to get the \r to show up in the data we have though 🤷 at least it will get removed if it does turn up!