Handling Non-Printable Characters and Blacklisting in Database Field Validation

Wes Wagner

Hi Xano community,

I have two questions around data field validation.

Recently, we undertook a penetration test which suggested we enforce stricter field validation, mostly to prevent cross-site scripting attacks. While Xano's "filter" feature has been helpful (see the bottom of this page on Xano's documentation), we have a couple of areas where we could use some additional guidance.

Specifically, we found that users were sending more obscure characters, such as a "non-breaking space" character (unicode \u00a0) and newline characters (\n), which were causing issues with our validation. With some research, we figured out how to whitelist the non-breaking space character: you can directly enter it into the whitelist (Option+Space on mac or Alt+0160 on windows), although it simply appears as a space.

However, we're still unsure how to add non-printable characters to the whitelist, such as the newline character.

My questions are:

1. Whitelisting non-printable characters: Is there a specific method for whitelisting non-printable characters like the newline character (\n)?
2. Character blacklisting: In most cases, blacklisting certain characters rather than whitelisting could be more beneficial for our use case. Is there a way to implement a blacklist in the current "prevent" filter or elsewhere in the system? From my understanding, the "prevent" filter is more about phrases than specific characters.

We've tried to search through the documentation and experiment on our own, but haven't been able to find specific answers to these queries. Any insights or guidance would be greatly appreciated!

Lachlan

HI @Wes Wagner

You could currently control this at the API layer with Regex probably being the most viable solution.

I've passed it on to the engineering team who may be able to look at having this added at the schema level.

Ray Deck

The Xano filters at the input validation level are easy to use but limited. For deeper validation I'd go to precondition checks inside the function stack. By running the input through filters, you get access to power tools, like the regex check Lachlan suggests.

When I am doing this kind of thing for complex validation I will often wrap the concern into a function that returns nothing but throws a precondition error to abort the endpoint. That separates the issue of validating the input from the rest of my stack. I only do this for my complex validation tasks - where I can go simple, I use simple. Where it has to be hard, I try to isolate so as not to make a mess of the business logic that follows!

Wes Wagner

This feels like a potential @Ray Deck question. 👀

Wes Wagner

Thank you, @Ray Deck and @Lachlan

Lachlan

@Wes Wagner, good news! It looks like we should have validation at the database schema level ready for our next release incase this is still a pending task for you to complete.

Wes Wagner

@Lachlan fantastic! Thank you for the update. We implemented a function on 100+ endpoints, but it's hardly a proper or thorough solution, so we'll definitely use this when it's released.