Recommandé
Recommandé
Contenu connexe
Tendances
Tendances (20)
En vedette
En vedette (20)
Similaire à Building Self-Defending Applications With OWASP AppSensor JavaOne 2016
Similaire à Building Self-Defending Applications With OWASP AppSensor JavaOne 2016 (20)
Dernier
Dernier (20)
Building Self-Defending Applications With OWASP AppSensor JavaOne 2016
- 1. Building Self-Defending Applications With OWASP AppSensor JavaOne 2016
- 2. me • appsensor dev lead (OWASP) • dev / security • why me? • twitter: @_jtmelton • email: jtmelton at gmail.com • github: jtmelton
- 3. agenda • history (recent) • motivations / problems • solution / tech • future / wrap-up
- 4. thesis: modern secure applications protect themselves against attackers
- 7. not too long ago dev • mostly web apps [RoR, PHP, .NET, Java) • ajax (jquery) use growing • mobile just getting started • deployment to VMs • hadoop picking up • BI tools • AWS starting • cloud hype cycle (NIST defines)
- 8. ~now dev • JS everywhere • functional / rx programming • cloud everything • ci/cd • nosql / CAP light • containers/orchestration • big data • stream processing • config management • iot • beacons [usage, ads, errors, performance] • actors/csp • microservices • cqrs / event sourcing • mobile
- 9. ~now dev • JS everywhere • functional / rx programming • cloud everything • ci/cd • nosql / CAP light • containers/orchestration • big data • stream processing • config management • iot • beacons [usage, ads, errors, performance] • actors/csp • microservices • cqrs / event sourcing • mobile 1 .. * of [scale, speed, cloud, lack of environmental access]
- 11. meanwhile … in security • 3rd party libs (dep-check) • bug bounties • sast / dast evolve (ZAP) • iast / rasp • http security headers • automatic encoding (JXT) • *-monkey -NetflixOSS • bdd-security/gauntlt • ci/cd plugins • 2fa • osquery 1 .. * of [scale, speed, cloud, lack of environmental access]
- 12. dev vs. security • dev is exploiting fundamental architectural and deployment changes to add business value • security is iterating on existing solutions - and - trying to close gaps (known problems)
- 13. motivations
- 14. traditional “security” • confidentiality and integrity important • availability often ignored by security (informs the whole industry- eg. tooling) • if availability important, runtime important
- 16. your environment • how many concurrent users do you have right now? • what are your users doing in the app?
- 18. Intuition: • “traditional” security, dev, ops doesn’t know what’s going on in the app at runtime (holistically)
- 19. Security defects are a subset of all defects
- 22. catching defects • what do dev/qa do for functionality? • test [unit, integration, system, manual, tools] • what do attackers do for security? • test [automated tools, manual]
- 25. observations • attackers do bad things • bad things often easily recognizable (to you … in your business … if you’re looking) • attacker success often* requires > 1 attempt * If not, you lose
- 26. Intuition: • security defects exist • attackers don’t magically know what’s vulnerable
- 27. Monitoring
- 29. Intuition: • existing (security) “monitoring” is usually terrible * * Note: a 2U box will not protect you
- 30. on people • 18.2 million devs • 200K security (all, not appsec only) • ~ 1.1 sec : 100 dev • 1.75 sec : 100 dev (bsimm)
- 31. Intuition: • there will never be enough “security” people
- 32. security modern dev • a single mature, static language • monolith • http (really html) endpoints • polyglot static and dynamic languages • microservices / soa • json, thrift, protobuf, grpc, etc. endpoints • WebAssembly ???
- 33. @petecheslock
- 34. Intuition: • “traditional” security tooling doesn’t fit modern dev
- 35. defender’s dilemma • attacker needs ONE successful attack • defender * must defend ALL attacks * you are defenders
- 36. in summary (so far) … • “traditional” security, dev, ops doesn’t know what’s going on in the app at runtime (holistically) • security defects exist • attackers don’t magically know what’s vulnerable • existing (security) “monitoring” is usually terrible • there will never be enough “security” people • “traditional” security tooling doesn’t fit modern dev … actual defense is _really_ hard
- 37. the pitch (a humble proposal)
- 38. having to deal with [scale, speed, cloud, lack of environmental access].. ..this as of now incomplete transition.. ..is an huge opportunity for improving security
- 39. the pitch (#0) • in addition to a secure SDLC … (ie. > 1 request/ attack) • if you’re not at this stage, work on it first
- 40. the pitch • figure out what’s happening at runtime X success AppSensor • make intrusion detection primitives available in app • exploit automated response > manual response • stop attacker before success • get self-protecting applications and valuable intel
- 41. X X X
- 42. terminology • event - suspicious • attack - malicious (1 .. * events) • response - take action (1 .. 1 attack) • detection point - activity category (e.g. cookie modification)
- 43. the tech
- 44. the tech • the architecture • getting data in (detection) • getting data out (visualization) • current efforts
- 45. Architecture
- 46. Application AppSensor 1. Event 2. Attack 3. Response
- 47. Application AppSensor 1. Event 2. Event 3. Attack 4. Response
- 48. AppSensor WAF NIDS App 1 App 2 App N Data Viz SIEM Analytics Events / Attacks Event / Attack / Response Notifications Policy Responses Correlation
- 49. AppSensor IN OUT Policy Reporting Engines Analysis Engines Listeners Event, Attack, Response StoresHandler Events / Attacks Responses
- 50. Emitters • ELK • CEF / Syslog • Influx / Grafana • WebSocket • JMX • Prometheus Framework Integration • Spring Security Configuration • XML
- 51. Execution Modes • REST • Kafka • ActiveMQ • RabbitMQ • Thrift • SOAP • embedded (jvm) Storage Providers • JPA2 • ElasticSearch • Mongo • Riak • Influx • File • In-memory (testing)
- 53. adding detection points • manually • appsensor-reverse-proxy • WAF (e.g. OWASP CRS in ModSecurity) • OWASP ASIDE (secure IDE plugin/ educational)
- 54. manual POST /account/transfer HTTP/1.1 Host: 127.0.0.1 User-Agent: Mozilla/5.0 (Win…) Accept: text/html,application/xhtml+xml Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Referer: http://127.0.0.1/account.php Cookie: PHPSESSID=l9…lgt5 Connection: keep-alive Content-Type: application/x-www-form-urlencoded Content-Length: 30 from_acct=xxx1234&to_acct=xxx9876&amt=20.00
- 55. manual POST /account/transfer HTTP/1.1 Host: 127.0.0.1 User-Agent: Mozilla/5.0 (Win…) Accept: text/html,application/xhtml+xml Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Referer: http://127.0.0.1/account.php Cookie: PHPSESSID=l9…lgt5 Connection: keep-alive Content-Type: application/x-www-form-urlencoded Content-Length: 30 from_acct=xxx1234&to_acct=xxx9876&amt=20.00
- 56. manual POST /account/transfer HTTP/1.1 Host: 127.0.0.1 User-Agent: Mozilla/5.0 (Win…) Accept: text/html,application/xhtml+xml Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Referer: http://127.0.0.1/account.php Cookie: PHPSESSID=l9…lgt5 Connection: keep-alive Content-Type: application/x-www-form-urlencoded Content-Length: 30 from_acct=xxx1234&to_acct=xxx9876&amt=20.00
- 57. manual @POST public Response transfer( String from, String to, String amount) { transfer(from, to, amount); return Response.ok(); }
- 58. manual @POST public Response transfer( String from, String to, String amount) { if ( currentUser.owns(from) ) { transfer(from, to, amount); } return Response.ok(); }
- 59. manual @POST public Response transfer( String from, String to, String amount) { if ( currentUser.owns(from) ) { transfer(from, to, amount); } else { showErrorPage(); // normal error handling } return Response.ok(); }
- 60. manual @POST public Response transfer( String from, String to, String amount) { if ( currentUser.owns(from) ) { transfer(from, to, amount); } else { appsensor.addEvent( new Event(currentUser, "ACE2") ); showErrorPage(); // normal error handling } return Response.ok(); }
- 61. recommendations • Aim for key architectural choke points • AOP can often be helpful • Exploit custom exception hierarchies • Look for business logic cases • Train developers to think this way
- 63. appsensor-reverse-proxy • written in go • blocks requests • canned detection points (toggle-able) • easily extendable • https://github.com/jtmelton/appsensor- reverse-proxy
- 64. WAF • Send events and/or attacks • Receive and process responses • OWASP CRS in ModSecurity has AppSensor rules already • https://www.trustwave.com/Resources/ SpiderLabs-Blog/Implementing-AppSensor- Detection-Points-in-ModSecurity/
- 65. OWASP ASIDE • secure programming IDE plugin (eclipse) • reminder icon or highlight • drop down list of applicable sensors • auto-insertion of ASIDE sensor APIs and code refactoring • UNCC SIS project (educational component) • https://www.owasp.org/index.php/ OWASP_ASIDE_Project
- 66. OWASP ASIDE
- 69. Viewing Data (getting data out)
- 70. viewing data • ELK stack (OWASP SoC) • influxdb / grafana (OWASP SoC) • appsensor-ui (video demo)
- 71. ELK
- 72. Influx / Grafana
- 80. Current Efforts
- 81. Rules Engine Goals • Expand detection capabilities by providing boolean logic and new span primitives • Reduce false positives by leveraging several suspicious events to discover a malicious event
- 82. Rules Engine • Multiple sensors grouped into single “Rule” to trigger an attack • Rule combines sensors with AND/OR/NOT/THEN operators • Thresholds can be lowered without increasing false- positive rate because there are multiple indicators • I.e. many SUSPICIOUS factors can define a MALICIOUS factor
- 83. Example - Default Engine Sensor1 - Multiple failed login attempts (50 attempts / 1 minute) Rule: Sensor1
- 84. Example - Rules Engine with AND Sensor1 - Multiple failed login attempts Sensor2 - Use of blacklisted characters Sensor3 - Password attempt too long Sensor4 - Multiple usernames attempted from single IP Rule: Sensor1 AND Sensor2 AND Sensor3 AND Sensor4
- 85. Example - Rules Engine with OR Sensor1 - Multiple failed login attempts Sensor2 - Use of blacklisted characters Sensor3 - Password attempt too long Sensor4 - Multiple users attempting to login from single IP Rule: Sensor1 AND (Sensor2 OR Sensor3 OR Sensor4)
- 86. Example - Rules Engine with NOT Sensor1 - Multiple failed login attempts Sensor2 - Use of blacklisted characters Sensor3 - Password attempt too long Sensor4 - Multiple users attempting to login from single IP Sensor5 - Blacklisted IP Rule: Sensor1 AND (Sensor2 OR Sensor3 OR Sensor4) AND NOT Sensor5
- 87. Example - Rules Engine with THEN Sensor1 - Use of blacklisted characters Sensor2 - Large file upload Sensor3 - Large file download Sensor1 THEN (Sensor2 OR Sensor3)
- 88. Ultimately Any Combination Will Work Sensor1 OR Sensor2 THEN Sensor3 AND NOT (Sensor4 OR Sensor5) THEN Sensor6 AND Sensor7 AND Sensor8 AND Sensor9 AND Sensor10
- 93. 1. Collect Events Sensor1 THEN Sensor2 Under the Hood 2. First Expression
- 94. 1. Collect Events Sensor1 THEN Sensor2 Under the Hood 2. First Expression
- 95. 1. Collect Events Sensor1 THEN Sensor2 Under the Hood 2. First Expression
- 96. 1. Collect Events Under the Hood Sensor1 THEN Sensor2 2. First Expression 3. Second Expression
- 97. Sensor1 THEN Sensor2 1. Collect Events Under the Hood 2. First Expression 3. Second Expression 4. Rule Triggered Thanks David Scrobonia!
- 98. GSoC 2016 (ML) • An external system using Logstash, Kafka and Spark that takes in log files, runs machine learning (ML) analysis on the features specified by user and generates a list of rules sorted by an evaluation criteria. • The aim of this system is to assist users to identify anomalous patterns or behaviors on their system in a readable manner.
- 99. ML Analysis • Currently implemented algorithms for both simple and complex analysis are k-means clustering, naïve bayes, logistic regression and decision trees. • You would need to write your own indexer for any new categorical features if the algorithm only accepts numeric features and your own vectorizer for different combinations of multiple features. • Simple analysis uses one feature (example: HTTP verb, response, lat/long) for clustering and classification • Complex analysis takes into account multiple features for the ML process.
- 100. ML Future Work • Project is definitely still a work in progress. • Some changes/improvements to be made: 1. Add support for more common log file formats 2. Add support for other features that can be used in a log file 3. Add visualization to allow users to understand results of complex analysis better • One major goal of current efforts is a tool you can send web logs in standard formats and receive “suggested rules”
- 101. ML Docs and Video • All documentation for the GSoC project can be found at: https://github.com/ timothy22000/GSoC-MLAnalysisEngine • https://youtu.be/tsdC_ftjF1g (video demo) • Thanks Timothy Sum Hon Mun!
- 102. Analysis Engines Simple thresholds Large user changes in user base or application Anomaly Detection Aggregation of simple thresholds Basic Trend* Machine Learning Rules
- 103. future plans
- 104. future • analysis engines (add trend, expand rules and ML) • expand reverse proxy • expand appsensor-ui • server assembler • framework integration for detection points (spring security exists, add others) • your idea here ???
- 105. you • help wanted! • plenty of places to contribute and improve • friendly, helpful community • https://github.com/jtmelton/appsensor/issues • https://www.owasp.org/index.php/ OWASP_AppSensor_Project#tab=Road_Map_a nd_Getting_Involved
- 106. wrap-up
- 107. ~related projects • repsheet • ensnare • fido • riemann • apache eagle • devsecops • elastalert • fouroneone • https://github.com/dschadow/ApplicationIntrusionDetection / • http://files.dominikschadow.de/event_javaone2015-con2022.pdf
- 108. pick a tool (or 2) … but use the idea
- 110. links • https://www.owasp.org/index.php/ OWASP_AppSensor_Project (download book, dev guide, etc.) • http://appsensor.org/ (end user / dev docs) • https://github.com/jtmelton/appsensor
- 111. ?