Can Xano handle NDJSON format?

Eduardo Magalhaes

Hi Xano community 🤩 ,

I am using Prowebscraper to scrape product and price data from two brazillian online grocery store and recently, when I added more scrapers in order to get more data points, I got some issues while I was trying to read this read with Xano.

I got an error message at Xano saying that the response I got from Prowebscraper's API did not provide a proper json file format (I tried to import it with the import function in Xano).

Than I asked Prowebsraper's support to fix that (give me a proper Json file format) and they repplyed to me saying the json format that they provide, specially when there is pagination in the scraper (which means, the scraper scrape data from more tham just one web page) is a NDJSON format.

So, I would like to ask Xano team if it is possible to read NDJSON format, so that I can hopefully 😁 read data from their API when there is pagination in one of my scrapers.

All the best,
Edu 🇧🇷

Shreyase

If the only newline characters inside the NDJSON text are the ones separating the JSON objects, it is most likely possible. 
Use the split filter on the text (NDJSON) and use the newline character as the separator. It should return a list of strings that can then be processed as JSONs. 

The key is to properly enter the newline character in the separator field. 
While editing the separator field, press Shift+Enter to open the text editor. Press Enter key once inside the editor so that you are able to see an empty line inside the editor and save it. 

Eduardo Magalhaes

Hi  , thanks for your hint. I will try to do that!

It seems that is my case, as you mentioned "If the only newline characters inside the NDJSON text are the ones separating the JSON objects, it is most likely possible. ".

Please, see an example of the API response with and NDJSON response.

[ndsjonexample.png] As soons as I try it, I will let you know if it has worked.

All the best,
Edu🇧🇷

Daniel Rjeili

Hello from brazil !  Just completing what  
JSON strings in NDJSON is separated with “new line” that is represented by a:  \n
👍try it

Shreyase

Hi , for some reason "\n" did not work for me. 
I should have added this separately. Apparently Xano's text fields are intended to accept 'text-as-it-is' without requiring any escape sequences like '\n'. The text editor that opens on pressing Shift+Enter is capable of accepting newline as a processed text (like text editors and word processors do) and does not process '\n' in the way we would expect it to. 

Shreyase

 It should work

Eduardo Magalhaes

Hey  , it worked! Thanks a lot! So happy!

Also thanks for the Shift enter hint, did not remember about that... 

All the best,
Edu