Applying sorting & filtering logic on "read-persisted" service

[Aditya Phatak]

Hi,

We have a requirement to fetch details from an external system using an API call on navigating to a view tab in a screen (e.g. we will create a new tab called “Documents” in Contact/Deal workspace). This call will be a real-time API integration (standard json approach for REST-ful API). The API will return list of details such as name, expiry date, document type, etc.

The requirement is to support “sorting” and “filtering” on this real-time data. We have an existing implementation where a similar requirement is handled using service-read persistence approach (but it doesn’t have sorting n filtering logic).

Can you please suggest how to approach the requirement? Should we persist all the details received from API to apply sort and filter? Or is there another way to handle this with read-persistence without involving DB persisted data?

 

Regards,

Aditya Phatak.

0

[nelson.yik]

I would think you would need to augment the service persistence to extend and apply sort/filter.

0

[Clayton Chow]

Typically, it is not recommended that the CRM application should sort and filter data that it is not the book of records for and is not persisting; Both for functional reasons (e.g. it may provide an inaccurate view of the data if it is stale) and for technical reasons (e.g. sorting will not be performant)

The implementation would depend on two main factors:
– the size of the data set
– capability of the API

The options available are:

1. Sorting and Filtering to be implemented by API
– typically RESTful APIs have built in functionalilty to support filtering (e.g. URL parameters) and pagination (e.g. offset paging).
– this reduces the risk of stale data being displayed to the user
– this solution keeps the NexJ application as a display vehicle

2. IF the complete result set is small (e.g. < 1000 records) and does not need to be saved between sessions, then the sorting can be done in the UI Framework
– the sorting and paging of the list can be done “in memory” if the data set size is small enough
– due to good design practices (e.g. data fidelity, efficiency of transferring data that is not used), filtering should still be implemented by the API even though it can be accomplished in the Persistance Mapping Read Script
– UI sorting can be achieved by going to the UI form and setting the Grid’s “paging” parameter to “none-sortserver”
– alternatively, the filtering and sorting can be done in the Persistence Mapping Read Script by manipulating the result set of the API call but is not recommended as mentioned above.
3. Have the filter and sort against a database-persisted class
– one type of implementation would be to have a job that pulls all the data from the API and persists it in CRM on a daily basis. Users can then read and filter against the CRM database which eliminates risk for sorting and filter (but it would not support a real-time requirement)
– another option would be to temporarily persist the API dataset for that particular Contact or Deal and then delete the data once the context changes.
– this has the highest overhead and risk as essentially it replicates the API data into the CRM database

0

[Author]

Posts