Get conversationId, messageId, and manifest in the response body from MHS Adapter

[robert-joscelyne]

The GP2GP message contains attributes contained outside of the message body that are required for successful processing.

These include:

• conversationId
• messageId
• manifest

[petervdm]

Linked to #163

[bartek-sarul]

@tomzo Does this mean that when a message is sent with those attributes, MHS adapter forwards it to spine but spine fails to process that message because of those missing attributes?
Where can we find example messages with those additional not-in-body attributes?

[tomzo]

What we've seen on the way from spine to MHS is that MHS extracts only the SOAP body of the messages and puts that on the queue. But what we would expect is to get all attachments and headers on the queue too.

Does this mean that when a message is sent with those attributes, MHS adapter forwards it to spine but spine fails to process that message because of those missing attributes?

I believe there is no way to send a message with extra attachments and headers to the MHS because it abstracts away the SOAP logic.

Where can we find example messages with those additional not-in-body attributes?

Have a look at example messages in https://github.com/nhsconnect/prm-deductions-gp2gp-adaptor/tree/master/docs/example_conversations/large-message-example-2 you may want to reach out to @fishey2 for more details on their origin.

[bartek-sarul]

@tomzo ok, now I understand that it's for the inbound service
Is it true that this is a duplicate of #163
or those two differ somehow?

[tomzo]

They are slightly different. Sorry, I should have placed that comment earlier in the #163

This one is really about how GP2GP messages look like. When foundation supplier sends back a message such as EHR, it will not contain the RefToMessageId field in the SOAP header. Instead it contains a ConversationId. A patch we did around it, is at tomzo@c42c9dc which is what we used to prove out in a PoC. It would be ideal to implement with a good test coverage.

Having manifest and messageid/conversationid attached to the messages placed on the queue would be a nice enhancement. That's the part that overlaps with #163

[bartek-sarul]

https://gpitbjss.atlassian.net/browse/NIAD-137

[bartek-sarul]

@robert-joscelyne can we get an example request that is sent to outbound and also an example of what is sent back to the inbound?

[fishey2]

Hi Bartek,

Prerequisites:

• Known Patient
• Retrieved Patient Details from PDS
• Updated ODS code to match where you want to the GP2GP response sent

GP2GP Request (RCMR_IN010000UK05)
Template Message: https://github.com/nhsconnect/prm-deductions-gp2gp-adaptor/blob/master/src/templates/ehr-request-template.js
Headers:
'Content-Type': 'application/json', 'Interaction-ID': interactionId, 'Sync-Async': false, 'Correlation-Id': conversationId, 'Ods-Code': odsCode, 'from-asid': config.deductionsAsid
Our MHS Client: https://github.com/nhsconnect/prm-deductions-gp2gp-adaptor/blob/master/src/services/mhs/mhs-outbound-client.js

Acknowledgement (MCCI_IN010000UK13): https://github.com/nhsconnect/prm-deductions-gp2gp-adaptor/blob/master/docs/example_conversations/patient/9442964410/gp2gp-acknowledgement.xml

GP2GP Response Example (RCMR_IN030000UK06): https://github.com/nhsconnect/prm-deductions-gp2gp-adaptor/blob/master/docs/example_conversations/patient/9442964410/gp2gp-response.xml

The messages are the raw messages that are received on the async forward reliable workflow, which we have moved directly onto the message queue.

Additional Resources:
PDS v3 Level 3 MIM: https://data.developer.nhs.uk/dms/mim/6.3.01/Domains/PDS/Document%20files/PDS%20IM.htm
GP2GP 1.1 MIM: http://prm-mim-3.1.10.s3-website.eu-west-2.amazonaws.com/
GP2GP 2.2: (Large Message (COPC)) MIM: http://prm-mim-7.2.00.s3-website.eu-west-2.amazonaws.com/

Large Message Provided Example 1: https://github.com/nhsconnect/prm-deductions-gp2gp-adaptor/tree/master/docs/example_conversations/large-message-example-1

Large Message Provided Example 2: https://github.com/nhsconnect/prm-deductions-gp2gp-adaptor/tree/master/docs/example_conversations/large-message-example-2

Note: The exact origin of the Large Message Response Examples is not known.

[bartek-sarul]

@fishey2 @robert-joscelyne
can you clarify those listed below?
I was using
https://github.com/nhsconnect/prm-deductions-gp2gp-adaptor/blob/master/docs/example_conversations/patient/9442964410/gp2gp-response.xml
as the example

1. I think that conversationId and messageId are already available in queue message properties as headers.
   
   After decoding from base64, we get:

�St����l�����
message-id�$31FA3430-6E88-11EA-9384-E83935108FD5��correlation-id�$5A36471B-036B-48E1-BBB4-A89AEE0652E1

which are the messageId and conversationId from the message itself

Is this enough or would you still like to have them in the message body as well as another 2 json fields?

2. In terms of adding the manifest, I understand you would like to whole manifest XML to be available as new field on the queue message body?
   For example if the manifest looks like:

<eb:Manifest SOAP:mustUnderstand="1" eb:version="2.0">
    <eb:Reference xlink:href="cid:payload@tpp-uk.com/SystmOne/GP2GP1.1A">
        <eb:Schema eb:location="http://www.nhsia.nhs.uk/schemas/HL7-Message.xsd" eb:version="2.0" />
        <hl7ebxml:Payload style="HL7" encoding="XML" version="3.0" />
    </eb:Reference>
    <eb:Reference xlink:href="cid:attachment1.0@test.com" eb:id="_31B75ED0-6E88-11EA-9384-E83935108FD5">
        <eb:Description xml:lang="en-gb">31B75ED0-6E88-11EA-9384-E83935108FD5_patient-attachment.txt.txt</eb:Description>
    </eb:Reference>
</eb:Manifest>

If both (1) and (2) are true, then the adjusted message body would look like as follows:

{
    "payload": "...",
    "attachments": [ ... ],
    "correlation_id": "5A36471B-036B-48E1-BBB4-A89AEE0652E1",
    "message_id": "31FA3430-6E88-11EA-9384-E83935108FD5",
    "manifest": "<eb:Manifest SOAP:mustUnderstand=\"1\" eb:version=\"2.0\" xmlns:eb=\"http://www.oasis-open.org/committees/ebxml-msg/schema/msg-header-2_0.xsd\" xmlns:hl7ebxml=\"urn:hl7-org:transport/ebxml/DSTUv1.0\" xmlns:xlink=\"http://www.w3.org/1999/xlink\"><eb:Reference xlink:href=\"cid:payload@tpp-uk.com/SystmOne/GP2GP1.1A\">........"
}

Notice that after extracting manifest node, namespaces have been added to it for end user to be able to parse the data correctly.

Apart from that I wonder, why only the manifest would need to be put in the body? Why not add the whole soap message? Any security reason?
If we do that, end user would be able to extract any piece of data he would like.
In the end what it mean is the incoming "bundle" will be just wrapped nicely in json on the queue without omitting any data.
Example:

{
    "payload": "...",
    "attachments": [ ... ],
    "correlation_id": "5A36471B-036B-48E1-BBB4-A89AEE0652E1",
    "message_id": "31FA3430-6E88-11EA-9384-E83935108FD5",
    "soap_message": "<?xml version=\"1.0\" encoding=\"UTF-8\"?><SOAP:Envelope xmlns:SOAP=\"http://schemas.xmlsoap.org/soap/envelope/\" xmlns:eb=\"http://www.oasis-open.org/committees/ebxml-msg/schema/msg-header-2_0.xsd\" xmlns:hl7ebxml=\"urn:hl7-org:transport/ebxml/DSTUv1.0\" xmlns:xlink=\"http://www.w3.org/1999/xlink\" xmlns:xsi=\"http://www.w3c.org/2001/XML-Schema-Instance\"><SOAP:Header>"
}

[fishey2]

@bartek-sarul

I completely agree with your suggestion above, where you provide the entire SOAP message. I do not see any security/any other reasons to not do that

I have a question related to the headers, and also #207: When you decode the headers (whether or not they originate from within the message body, or as a header) they do not seem to be in a readable format and contain escape/control characters:

�St����l�����
message-id�$31FA3430-6E88-11EA-9384-E83935108FD5��correlation-id�$5A36471B-036B-48E1-BBB4-A89AEE0652E1

Ideally, it'd be great if the items were added as individual headers/properties (as demonstrated below), although, we can just as easily extract this from the SOAP Envelope, so happy either way.

properties: {
    "message-id": "31FA3430-6E88-11EA-9384-E83935108FD5",
    "correlation-id": "5A36471B-036B-48E1-BBB4-A89AEE0652E1",
    "JMS_AMQP_NATIVE": true,
    "JMS_AMQP_HEADER": true,
    "JMS_AMQP_ContentType": "multipart/related; boundary=\"--=_MIME-Boundary\"; type=\"text/xml\; charset=\"UTF-8\"; start=\"soappart\"",
    "JMS_AMQP_PROPERTIES": true
}

[bartek-sarul]

@fishey2
I've checked those message properties

�St����l�����
message-id�$31FA3430-6E88-11EA-9384-E83935108FD5��correlation-id�$5A36471B-036B-48E1-BBB4-A89AEE0652E1

this is how it looks like after decoding manually but if you use AMQP client then it's all decoded properly

I'll leave them as they are now as additional properties and won't include in body

[fishey2]

@bartek-sarul

Ok, that would help, thanks.

Just for your information, we were seeing this when using a STOMP client (stompit) instead of an AMQP client with ActiveMQ.

[bartek-sarul]

@fishey2
OK I'll keep an eye on that