Why is data validation important? What are effective ways to ensure data is valid? In this session we'll explore how data validation is directly linked to security, stability and developer productivity when dealing with untrusted or unknown data sources. We'll discuss the dangers of code that does not validate its data - everything from injection to DOS attacks. We'll go hands on with joi (https://github.com/hapijs/joi) and Express (http://expressjs.com/) to see how data validation can make code easier to work with. No more "Uncaught ReferenceError" or if null checks littered around the code base. In the end, we'll see how code can be secure, stable and magically awesome to work with.

2. Take Data Validation
Seriously
Paul Milham, WildWorks

3. WildWorks

4. Animal Jam

5. Outline
• Attacks
• Data Validation => Security
• Data Normalization => Stability
• Joi
• Tean
• Express Integration
• Hapi Integration
• Questions

6. Safety
• My job is to keep kids safe.
• How do we keep our application safe?
• Safe from what?

7. Attacks
• The web is full of jerks
• https://www.owasp.org/index.php/Category:Attack
• Read that for a bedtime horror story

8. SQL Injection
console.log(name); // paul
console.log(email); // '); DROP TABLE db.user; --
mysql.query(`INSERT INTO db.user (name, email) VALUES ('${name}', '${email}')`);

9. Shell Injection
console.log(pass); // "; rm -rf /"
require("child_process").exec(`
php -r "print crypt('${pass}','$1$rounds=1$salt$');"
`, (err, stdout, stderr) => {
});
// hopefully you're using containers

10. ReDOS
const msg = 'foo=bar' + ';'.repeat(65535) + 'domain=example.com';
console.time("regex");
console.log(msg.search(/;+$/));
console.timeEnd("regex"); // regex: 5854.071ms :(
• This is a sample vulnerability in tough cookie
• https://snyk.io/vuln/npm:tough-cookie:20160722
• Be careful of "evil" regex

11. Security
• It’s a scary world
• Security is important
• There’s a lot more than just those three

12. Validation
• Verify the shape of the data
• Malicious data can’t get in
• First line of defense

13. Simple Joi
"use strict";
const Joi = require("joi");
Joi.validate("srsly a string", Joi.string(), (err, value) => {
console.log(err); // null
console.log(value); // "srsly a string"
});

14. Joi Failure
Joi.validate(5, Joi.string(), (err, value) => {
console.log(err); // Error
console.log(value); // 5
});

15. Joi Schema
const schema = Joi.object().keys({
username: Joi.string().email({tldWhiteList: ["wildworks"]}).required(),
password: Joi.string().min(6).max(25).required(),
});
Joi.validate({
username: "paul.milham@wildworks.com",
password: "justinbieber",
}, schema, (err, value) => {
console.log(err);
console.log(value);
});

16. All In
const schema = Joi.object().keys({
username: Joi.string().email({tldWhiteList: ["wildworks"]}).required(),
});
Joi.validate({
username: "paul.milham@wildworks.com",
password: "justinbieber",
}, schema, (err, value) => {
console.log(err); // justinbieber is not allowed
});

17. All In
• Validating one field means validating them all
• Hard for devs to forget

18. Data Normalization
• Normalization is being a good citizen
• Normalization creates a contract with your consumer
• Normalization goes a lot deeper than this (we'll get to that later)

19. Joi Conversion
Joi.validate("1.916", Joi.number(), (err, value) => {
console.log(value.toFixed(1)); // 1.9 (No TypeError!)
});

20. Joi Defaults
Joi.validate(undefined, Joi.number().default(0), (err, value) => {
console.log(value.toFixed(1)); // 0.0 (No TypeError!)
});

21. Tean
•Declarative syntax (schemas are POJOs)
•Async
•Convert data into models
•Strict by default
•https://www.npmjs.com/package/tean
•Note that custom validators were recently added to Joi

22. Tean Validation
// simple validation
tean.object({animal: "string"}, {animal: “kangaroo”},
(isValid, result) => {
console.log(isValid); // true
console.log(result); // {animal: "kangaroo"}
});

23. Tean Failure
tean.object({animal: "string"}, {animal: null}, (isValid, result) => {
console.log(isValid); // false
console.log(result); // ["animal (null) is not a string"]
});

24. Tean Normalization
// optional parameters
tean.object({animal: “string(kangaroo,tiger)=tiger”, sparkles: "bool=true"}, {animal: "tiger"},
(isValid, result) => {
console.log(isValid); // true
console.log(result); // {animal: "tiger", sparkles: true}
// Note that the original object is not altered! Normalized and validated data is passed into
"result" in the callback
});

25. Model Mapping
tean.object(req.body.params, {
accessory: "avatarAccessory",
user: ["userUid"],
}, (isValid, result) => {
});

26. Data Normalization
• Provides a friendly API
• Provides consistency and reliability
• Eliminates lots of common bugs

27. Express
• Everyone uses it!
• No built in validation!
• Too many exclamation points!
• https://expressjs.com/

28. Express + Joi
app.get('/:username', function (req, res) {
const schema = Joi.object().keys({
username: Joi.string().required(),
});
Joi.validate(req.params, schema, (err, value) => {
console.log(err);
req.params = value;
res.send(`${req.params.username} is the best!`);
});
});

29. Express + Tean
app.get('/:user', function (req, res) {
tean.object(req.body.params, {
user: "userUid",
}, (isValid, result) => {
console.log(isValid);
req.params = result;
res.send(`${result.user.name} is the best!`);
});
});

30. Problem
• We’re relying on the developer to remember to validate
• This is a problem for maintenance and updates
• Middleware to the rescue!

31. Route Middleware
this.app.post(options.route,
tean.expressRequest(options.paramMap),
(req, res) => {
// do stuff
options.handler(req.safeData, req, res);
},
(err, req, res) => {
console.log(err.stack);
res.status(500).send();
}
);

32. Express + Joi
• https://www.npmjs.com/package/celebrate

33. Hapi
• Hapi isn't minimalist like Express
• Lots of options out of the box
• http://hapijs.com/

34. Hapi Validation
app.route({
method: "POST",
path: "/",
config: {
handler: (req, reply) => {
reply("hey!");
},
validate: {
payload: {
username: Joi.string().email().required(),
password: Joi.string().max(25).required(),
},
},
},
});

35. Take Away
• FORCE validation of data - an opt in system isn't good enough
• Make sure shape of data is acceptable
• No validation, no data
• This ensures malicious data does not enter your application

36. Take Away
• FORCE normalization of data shape
• Data should always have a consistent shape
• Make data access and usage reliable
• Eliminates lots of “stupid” bugs

37. On the Way Out
• Have you thought about data security on the way out?
• Mind blown!
• Prevent Data Leaks from "heartbleed" or SQL Injection
• Provide same stability contract for your client app

38. Thanks!
• Any questions?
• @domrein