Skip to Main Content
  • Questions
  • How to consume external API with JSON response and write to Oracle DB hosted on AWS RDS

Breadcrumb

Question and Answer

Chris Saxon

Thanks for the question, Nick.

Asked: January 30, 2017 - 11:45 pm UTC

Last updated: February 01, 2017 - 2:54 pm UTC

Version: 12c

Viewed 10K+ times! This question is

You Asked

Hi,

I am looking for options and best practices for the following use case:

- consume an external API every 15 seconds, which returns JSON data and insert the response into an oracle db table
- the JSON response has around 80 fields which need to be mapped to oracle datatypes with a mix of boolean, string and number fields
- I have created a simple utl_http request which returns the JSON in string format but I am stuck on how to best proceed for mapping the returned data string
- the procedure needs to also remember the last insert id to add as a parameter in the api url so that only new records are returned

Questions:

1. Is it best to run a PL/SQL Procedure with Scheduler or would API manager be better suited?
2. Is there a size limit to the JSON response?
3. Are there plugins required to map JSON to oracle datatypes. If so are there options to add these plugins to AWS RDS hosted oracle db?

Any help you can provide would be greatly appreciated.

and Chris said...

1. If you're happy to have your database make call over the internet, you can consume REST services using PL/SQL. You can find examples at:

https://asktom.oracle.com/pls/apex/f?p=100:11:::YES:RP:P11_QUESTION_ID:9529251800346874328
https://technology.amis.nl/2015/05/11/invoke-a-rest-service-from-plsql-make-an-http-post-request-using-utl_http-in-oracle-database-11g-xe/

I'm not sure what you mean by API manager, so I can't comment on the differences.

2. The read procedures return varchar2. So if you want to get JSON documents bigger than 32k you'll need to read them into a clob. Tim Hall has an example for saving webpages at:

https://oracle-base.com/articles/misc/retrieving-html-and-binaries-into-tables-over-http

3. Assuming you're on 12.1.0.2, you can store the JSON document in a table. Then you can use JSON* functions to extract parts of it, for example:

create table t( doc_id  int, 
 doc_details varchar2(1000)
);

insert into t values (1 , '{"id":1, "name" : "Jeff"}' );
insert into t values (2 , '{"id":2, "name" : "Jane", "status":"Gold"}' );
insert into t values (3 , '{"id":3, "name" : "Jill", "status":["Important","Gold"]}' );
insert into t values (4 , '{"name" : "John", "status":"Silver"}' );
commit; 

select json_value(doc_details, '$.name') from t;

JSON_VALUE(DOC_DETAILS,'$.NAME')  
Jeff                              
Jane                              
Jill                              
John


Read more about these at:

https://blogs.oracle.com/developer/entry/series_json_within_the_oracle
https://oracle-base.com/articles/12c/json-support-in-oracle-database-12cr1

There's more JSON functionality available in 12.2. In the meantime, the libraries APEX_JSON (part of APEX 5) and PL/JSON fill in some of the gaps:

https://docs.oracle.com/cd/E59726_01/doc.50/e39149/apex_json.htm#AEAPI29635
https://github.com/pljson/pljson

Rating

  (1 rating)

Is this answer out of date? If it is, please let us know via a Comment

Comments

Nick Mead, February 01, 2017 - 12:26 am UTC

Thank you for such a detailed reply. Very helpful with the links provided.

Just for clarity, the API Manager I was referring to is as seen here: http://docs.oracle.com/cloud/latest/apimancs_gs/OSBAM/GUID-07B22F10-7674-4BAA-90EA-8E6431942F22.htm#OSBAM107

Is this service only for cloud based instances?
Is it only used for exposing API access to the DB itself, rather than hitting external Restful APIs?
Chris Saxon
February 01, 2017 - 2:54 pm UTC

Gotcha, that's part of Fusion Middleware. I'm not familiar with it, but from the preface:

This document is intended for curators, consumers, and administrators of APIs in Oracle API Manager, developers who create proxy services in Oracle Service Bus, and anyone interested in learning about the product.

It sounds like it's for managing RESTful services in general.

If you're looking to make database REST API, ORDS fits the job:

http://www.oracle.com/technetwork/developer-tools/rest-data-services/overview/index.html

More to Explore

CLOB/BLOB/etc

Complete documentation on Securefiles and Large Objects here