Overview

This section outlines the various methods service consumers can request operations and/or data. Basically the SIF infrastructure allows two distinct ways of requesting operations and data. These are referred to as:

• IMMEDIATE Request/Response (synchronous message pattern) which is the default way any REST API would process a HTTP request.

• DELAYED Request/Response (asynchronous message pattern).

Each message pattern has its distinct and useful application. The two message patterns and where they might be used (applicable) are detailed in the next few sections.

Assumptions & Constraints

It is important to note that DELAYED Request/Response functionality is only available for batch operations. DELAYED Request/Response is not supported for single object operations.

IMMEDIATE Request/Response

A Service Consumer invokes an operation (makes a Service Provider Request) via an HTTP request to the request or service connector service (as listed in the applicable infrastructureService node of the service consumer's environment XML/JSON).  The request is evaluated, and the specified service name (ex: Student), zoneId (specified or default) and contextId (specified or default) are used to determine the correct registered service provider endpoint to receive it.  If no service provider corresponds to these service-scoping parameters, the request must be rejected with an error code of 404 - Not found with an optional SIF error message payload.

Otherwise the consumer request is routed to the assigned URL of the selected service provider, and the response is immediately (synchronously) returned in the HTTP Response, as shown in the figure below.

Open

Note:

• The Request/Service Connector as well as the Provider Registry Service can be part of or be provided by a Broker, can be modular services or could potentially be part of a provider (often in DIRECT environments).

• In some basic DIRECT environments there might be no provider registry service at all.

• A Service Consumer may provide a HTTP header called requestType with a value of "IMMEDIATE" to indicate to the Request/Service Connector that an IMMEDIATE request/response is invoked. This is also the default behaviour if that HTTP header is not provided.

Error Management

If an IMMEDIATE request/response is not supported in an environment the Request and Service Connector services must return an HTTP error status code of 405 - Method not allowed. Optionally an appropriate error message payload can be returned in the response.

Applicability

IMMEDIATE request/response is the most “natural” message pattern for any REST API. It is commonly used in web-applications, mobile applications where an end-user application requests data or interacts with an end-user. It is generally not an ideal message pattern where complex and time consuming operations need to be performed on the provider or where large data set processing is requested by a service consumer.

DELAYED Request/Response

The SIF Infrastructure (v3.0+) specifies functionality for DELAYED requests. This functionality allows an asynchronous message pattern. Until SIF Infrastructure 3.4 the “asynchronousity” was only available for service consumers. In the SIF Infrastructure 3.5 the “asynchronous” behaviour has been extended to service providers. The service provider can support IMMEDIATE and/or DELAYED request/response. Depending on the “mode” of the service provider different infrastructure components are required by the service provider and the service provider has some more responsibilities in the request/response message flow. The next two sections describe the two options, both of which are the same for a service consumer but slightly different for a service provider. The two available options are:

• DELAYED Consumer - IMMEDIATE Provider (since SIF 3.0 Infrastructure)

• DELAYED Consumer - DELAYED Provider (since SIF 3.5 Infrastructure)

As highlighted, the service consumer side of a DELAYED request/response is the same regardless of the mode the service provider acts in. The consumer is expected to have a FIFO queue created already via thehttps://a4ldocumentation.atlassian.net/wiki/spaces/ARCHITECTU/pages/156270623. For each DELAYED request the service consumer must provide the ID of that queue as a HTTP Header called queueId in the HTTP request. Responses to the DELAYED request will be published to that queue. The service consumer can then poll this queue for responses via the https://a4ldocumentation.atlassian.net/wiki/spaces/ARCHITECTU/pages/156270623.

Note:

DELAYED Request/Response functionality is only available for batch operations. DELAYED Request/Response is not supported for single object operations.

DELAYED Consumer - IMMEDIATE Provider (since SIF 3.0 Infrastructure)

This section describes the functionality of a DELAYED request/response as it is available since SIF 3.0 Infrastructure. It is important to note that in this case the service provider isn’t really in a “DELAYED” mode and therefore cannot act in a true “asynchronous” manner. The DELAYED message pattern only applies to the service consumer. The image below illustrates that behaviour.

Open

1. A consumer issues a DELAYED request to the request or service connector.

2. The DELAYED request is received by a request or service connector and HTTP Status code of 202 - Accepted must be returned to the service consumer’s request. The consumer can now go back and do other things.

3. The request or service connector takes that DELAYED request and invokes the provider via a standard HTTP request. Note that the request or service connector may look up the provider registry (not shown in the illustration above) to determine where the request shall be forwarded to.

4. As per standard HTTP the connection between the request or service connector and the provider remains open until the provider has served the request and has provided a response.

5. Once the request or service connector has a response it will put it onto a consumer owned queue.

6. The consumer will poll its queues via thehttps://a4ldocumentation.atlassian.net/wiki/spaces/ARCHITECTU/pages/156270623 for responses and once one is received it will process it.

Step 3 & 4 above is a “synchronous” message pattern! It means that the request to the provider remains open until the provider has fully processed it and has responded with an appropriate message.

Requirements for Request and Service Connector

The request and service connector must provide a number of HTTP Headers to the consumer’s designated response queue when it relays a response it retrieved from the service provide to the response queue. These HTTP headers are important for a consumer to correlate a delayed response with the original request. The minimum list of such HTTP Headers are:

• messageType

• navigationPage (assuming it was a paged request)

• navigationPageSize (assuming it was a paged request)

• navigationId (assuming it was a paged request)

• requestId (if provided by service consumer)

• relativeServicePath

• responseAction (note this should be the same as the requestAction)

• serviceType

Unbound Paged Queries

For unbound paged queries the request and service connector have some additional responsibilities to meet the correct behaviour expected by the consumer. An unbound paged queries is a query where a consumer requests all objects to be returned but each response shall hold a maximum number of objects. This is indicated by the service consumer by providing only the navigationPageSize HTTP header in the request (no navigationPage HTTP Header, though).

Expected behaviour of request and service connector

The respective connector checks if only navigationPageSize the HTTP header is set. If so, it is the responsibility of the respective connector to call the service provider multiple times, each time with an incremented navigationPage HTTP header (starting with navigationPage=1). It must repeat that call until the service provider returns no more data (HTTP status code 204). For each call the respective connector must put the response onto the consumer’s designated queue as described in previous paragraph.

DELAYED Consumer - DELAYED Provider (since SIF 3.5 Infrastructure)

This section describes the functionality of a DELAYED request/response as it is available since SIF 3.5 Infrastructure. It is important to note that in this case the service provider can operate in a true DELAYED (asynchronous) mode. This is the key difference between the option described in the previous section and this section.The image below illustrates that behaviour.

1. A consumer issues a DELAYED request to the request or service connector.

2. The DELAYED request is received by a request or service connector and HTTP Status code of 202 - Accepted must be returned to the service consumer’s request. The consumer can now go back and do other things. 

3. The request or service connector takes that DELAYED request and puts it on the provider’s “pending request” queue.

4. Provider polls its “request queue”.

5. The Provider will get a pending request and the connection between the queue and provider is closed. Now the provider starts processing the request.

6. As it processes the request it will produce response messages and send (HTTP POST) them one by one to the Response Connector endpoint.

7. The Response Connector will respond to the service provider with an HTTP Status code of 202 - Accepted, indication that it got the response message(s). 

8. The Response Connector will put the responses onto a consumer owned queue.

9. The consumer will poll its queues via the for responses and once one is received it will process it.

It is important to note that in this true DELAYED mode that there are no “long-lasting” open connections. Both, the service consumer and service provider can process messages at their leisure, i.e. when they are ready, and there is no longer a potential issue of connection timeouts.

Requirements for Request and Service Connector

The request or service connector knows whether a DELAYED request can only be served by a DELAYED provider by inspecting the provider registry. If the requested operation (i.e CREATE) is marked as “DELAYED” in the provider registry then the consumer request MUST be put onto the provider’s request queue in its entirety. This includes all URL details, HTTP Headers, payloads where applicable and the HTTP verb (GET, POST etc.). To do so the service consumer’s request must be mapped to a “queue message” that can be stored on a queue. The following parts need to be mapped to a “queue message”:

HTTP Verb

Note that the HTTP verb needs to be translated into a HTTP Header called “requestAction” if it is not given by the service consumer. Special care must be taken for QBE and Batch Delete where the HTTP verb is different to the actual “requestAction”. QBE is conveyed by a HTTP POST but the requestAction must be translated to “QUERY”. A Batch Delete is conveyed by a HTTP PUT but the requestAction must be translated to “DELETE”.

HTTP Headers

All HTTP Headers must be stored with the request on the service provider request queue as is.

Request URL

The consumer’s request URL must be translated to a  HTTP Header called “relativeServicePath”. The definition of that HTTP Header is defined here. This HTTP Header content will provide the service provider with all the information it needs that are normally available to the service provider for IMMEDIATE requests (i.e. zoneId, contextId etc.).

Payload

If a request contains a payload it must be stored onto the service provider’s request queue “untouched”, meaning the payload is not cracked open or inspected by the request connector.

Requirements for Service Provider

Queue Management

To enable full flexibility a subscribe model is used, similar to the subscribe model used by service consumers for events. Specifically this means that a service provider can:

• Create any number of queues

• Can establish subscriptions to manage which request shall be directed to which queue. This will provide the request or service connector sufficient details to determine on which provider queue a service consumer request shall put on.

Queue Creation

A service provider must use the to create queues in the same manner as a service consumer creates queues. Note that the same API is used by the service provider to read messages (requests) from a queue.

Queue Subscriptions

The is used by a service provider to configure its subscriptions for delayed requests. It is important that the “messageType” element in the subscription payload is set to “REQUEST” to indicate that the subscription is for provider requests.

Request Retrieval

The service provider must use the to read “pending requests” from its queues in the same manner as a service consumer reads “pending responses” from its queues (get next message). All the required request details are stored in the response of the “Get Next Message” in either HTTP Headers or where applicable in the payload.

Response Routing

Delayed responses generated by the service provider must be provided (HTTP POST) to the  “responsesConnector” endpoint. The mandatory HTTP Header “messageType” must be set to “RESPONSE” or “ERROR” in case any errors are to be returned in the response.

Response Queue ID

The service consumer’s queueId of the consumers response queue forms a critical link in the DELAYED request/response pattern. It indicates the queue where responses from the service provider shall be stored on.

Service Consumer

A service consumer is required to provide a queue id to the request connector or service connector endpoint for DELAYED requests. It is expected that all responses from a service provider are put onto this queue. The queue id is conveyed as a HTTP Header with the name queueId. 

Request & Service Connector

The queueId from the consumer’s request needs to be persisted on the applicable provider request queue and passed along to the service provider when the request is retrieved by the service provider from the queue. The service provider will require that queueId when it provides responses back to the response connector endpoint. If the queueId HTTP Header is not provided by the service consumer then a HTTP Status Code 412 - Precondition failed and optional error message must be returned to the service consumer:

Service Provider

A service provider must provide/echo the HTTP Header called queueId, that it has retrieved from the initial request, in each response it provides to the response connector. This ensures that the response connector knows on which consumer queue it must put the responses. There are several other HTTP Headers that must be provided by the service provider to ensure that the service consumer who issued the initial request can reconcile the responses with the initial request. These HTTP headers are:

• messageType

• navigationPage (assuming it was a paged request)

• navigationPageSize (assuming it was a paged request)

• navigationId (assuming it was a paged request)

• queueId (see above paragraph)

• requestId (if provided by service consumer)

• relativeServicePath

• responseAction (note this should be the same as the requestAction)

• serviceType

Unbound Paged Queries

For unbound paged queries DELAYED service providers have some additional responsibilities to meet the correct behaviour expected by the consumer. An unbound paged queries is a query where a consumer requests all objects to be returned but each response shall hold a maximum number of objects. This is indicated by the service consumer by providing only the navigationPageSize HTTP header in the request (no navigationPage HTTP Header, though).

Expected behaviour of service provider

The service provider executes the query request and will provide the result in pages of the requested size (navigationPageSize). It must send each ‘page’ with the max size of the consumer’s navigationPageSize HTTP header to the response connector endpoint. It must also echo the navigationPageSize HTTP header with the value of the number of records. This number may be lower for the last page. Additionally the provider must echo the requestId from the consumer so that the consumer can correlate all responses for a paged query.

Applicability

DELAYED Request/Response provide a very scalable approach to serve consumer requests where the request processing might be complex and may take a lengthy period of time. Because no HTTP connections remain open until the request is fully processed the likelihood of connection timeouts is greatly reduced. It further avoids the need to open up ports so that a connector can “POST” requests into a network rather service providers will poll requests (using HTTP GET) from their respective queue.