Tuesday, 22 August 2017

ZAP Browser Launch

We have just released a new feature for ZAP that allows you to launch browsers from within ZAP. The browsers are automatically configured to proxy via ZAP and ignore certificate warnings, making it much easier for people to get started with ZAP as well as for more experienced users who want to use ZAP with a variety of browsers. You can install and use Browser Launch right now via the ZAP Marketplace, which can be accessed via the 'Manage Add-ons' button in ZAP:



just 'Check for Updates':

and 'Update All':

You will now get a new 'Launch Browser' option in the Quick Start tab:


To see a demo of this feature see the following video:



Note that you must be using the latest version of ZAP (currently 2.6.0) and at least Java 8.
Version 2.6.0 does support Java 7, but this functionality requires Java 8. It’s worth noting that the next full release of ZAP will also require Java 8 as a minimum.

If you have any problems or questions about this new feature then please head over the the ZAP User Group.

Monday, 19 June 2017

Scanning APIs with ZAP

The previous ZAP blog post explained how you could Explore APIs with ZAP.
This blog post goes one step further, and explains how you can both explore and perform security scanning of APIs using ZAP from the command line.
This allows you to easily automate the scanning of your APIs.

Following the approach taken by the Baseline Scan we have introduced a new API scanning script which has only one dependency – Docker. You don’t need to install either ZAP or Java.
The script, zap-api-scan.py is included in the Weekly and Live ZAP Docker images, it will also be included in the next Stable image.

To use the API scanning script you just need to use the commands:
    docker pull owasp/zap2docker-weekly
    docker run -t owasp/zap2docker-weekly zap-api-scan.py -t \
        https://www.example.com/openapi.json -f openapi

By default the script:
  1. Imports the API definition supplied
  2. Actively scans the API using a custom scan profile tuned for APIs
  3. Reports any issues found to the command line
If no issues are reported then that does not mean that your API is safe.
If your API is particularly important or sensitive then it would be sensible to follow the scan up with a manual penetration test. You should also test the applications that use the API as data returned via the API could still be used to attack the application if it does not suitably escape data that has been originally entered via a user.

Command Line Options


The script has a number of command line options that allow it to be tuned to your requirements:

Usage: zap-api-scan.py -t <target> -f <format> [options]
    -t target         target API definition, OpenAPI or SOAP, local file or URL, eg https://www.example.com/openapi.json
    -f format         either openapi or soap
Options:
    -c config_file    config file to use to INFO, IGNORE or FAIL warnings
    -u config_url     URL of config file to use to INFO, IGNORE or FAIL warnings
    -g gen_file       generate default config file(all rules set to WARN)
    -r report_html    file to write the full ZAP HTML report
    -w report_md      file to write the full ZAP Wiki(Markdown) report
    -x report_xml     file to write the full ZAP XML report
    -a                include the alpha passive scan rules as well
    -d                show debug messages
    -P                specify listen port
    -D                delay in seconds to wait for passive scanning
    -i                default rules not in the config file to INFO
    -l level          minimum level to show: PASS, IGNORE, INFO, WARN or FAIL, use with -s to hide example URLs
    -n context_file   context file which will be loaded prior to scanning the target
    -p progress_file  progress file which specifies issues that are being addressed
    -s                short output format - dont show PASSes or example URLs
    -z zap_options ZAP command line options e.g. -z "-config aaa=bbb -config ccc=ddd"



Scan Rules


By default the script will use a Scan Policy tuned for APIs.
This disables rules that are focused on client side (e.g. browser) issues, such as the ones for detecting Cross Site Scripting, and also adds 2 additional rules that are implemented as scripts:
You can change which rules are run and how failures are reported using a configuration file. This allows you to tune the scanning script to meet your requirements for each of your APIs.
To generate a configuration file use the ‘-g’ option. This will create a file which includes all of the active and passive scan rules available: https://github.com/zaproxy/zaproxy/wiki/ZAP-API-Scan#configuration-file. You can edit this file using a text editor.
Changing a passive rule will only affect how failures are reported, but changing an active rule to IGNORE will prevent the rule from running. This is to reduce the overall scan time – passive rules are very quick while active rules can take a significant amount of time.

Specifying Values


ZAP will use a set of default values when importing APIs. In some cases these will not be suitable values for a specific application, and therefore will not exercise enough of the code. For example a username of “test” might not cause a new user to be created as it is not a valid email address.
For APIs defined using OpenAPI/Swagger you can specify the values you want ZAP to use via ZAP command line options.
For example the options:
  -config formhandler.fields.field\(0\).fieldId=username \
  -config formhandler.fields.field\(0\).value=test@example.com \
  -config formhandler.fields.field\(0\).enabled=true \
  -config formhandler.fields.field\(1\).fieldId=phone \
  -config formhandler.fields.field\(1\).value=012345678 \
  -config formhandler.fields.field\(1\).enabled=true

Will supply the following values to the named fields:
  username -> test@example.com
  phone -> 012345678

Support for specifying values for APIs defined using SOAP is also planned – if you need this then get in touch with the ZAP development team and we will do our best to prioritize this.

Note that as these are ZAP command line options you will need to specify them to the script using the -z script option.
If you need to specify lots of options then you can put them all in a property file, eg called options.prop
You can then run the API scan using a command like:

docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-weekly zap-api-scan.py \
    -t https://www.example.com/openapi.json -f openapi \
    -z "-configfile /zap/wrk/options.prop"


The "-v $(pwd):/zap/wrk/:rw" is a Docker option which maps the current working directory to a folder called /zap/wrk in the Docker instance.

Authentication


Some of your APIs may be protected using authentication mechanisms.
For mechanisms that use header values we recommend that you obtain suitable tokens for your application using whatever means are appropriate and then tell ZAP to use them via another set of command line options.
For example the options:
  -config replacer.full_list\(0\).description=auth1 \
  -config replacer.full_list\(0\).enabled=true \
  -config replacer.full_list\(0\).matchtype=REQ_HEADER \
  -config replacer.full_list\(0\).matchstr=Authorization \
  -config replacer.full_list\(0\).regex=false \
  -config replacer.full_list\(0\).replacement=123456789 \
  -config replacer.full_list\(1\).description=auth2 \
  -config replacer.full_list\(1\).enabled=true \
  -config replacer.full_list\(1\).matchtype=REQ_HEADER \
  -config replacer.full_list\(1\).matchstr=AnotherHeader \
  -config replacer.full_list\(1\).regex=false \
  -config replacer.full_list\(1\).replacement=abcdefghi

will cause the following headers to be added to every request ZAP makes:
  Authorization: 123456789
  AnotherHeader: abcdefghi

You can specify as many headers as you need to by using incrementing indexes.

This functionality is provided by the Replacer add-on included by default with ZAP. It is very powerful and can do much more than just inject new header values, so if you need to manipulate the requests ZAP makes in other ways then this could be a very good option for you.

Conclusion


The API scanning script is an easy way for you to automate security scanning of APIs defined using OpenAPI/Swagger or SOAP.
It can be used ‘out of the box’ or quickly tuned to meet your requirements via simple command line and configuration file options.

A future blog post will explain how you can automate the security scanning of HTML based web applications in a very similar way.

Monday, 3 April 2017

Exploring APIs with ZAP

APIs can be challenging for security testing for a variety of reasons.
The first problem you will encounter is how to effectively explore an API - most APIs cannot be explored using browsing or standard spidering techniques.
However many APIs are described using technologies such as:

These standards define the API endpoints and can be imported into ZAP using 2 optional add-ons.

Installing the add-ons

In order to import the API definitions you will need to add the relevant add-ons from the ZAP Marketplace.

To do this via the UI:


  1. Click on the ‘Manage Add-ons’ button
  2. Select the ‘Marketplace’ tab
  3. Click on the ‘Check for Updates’ button if no add-ons are shown
  4. Select and install the add-ons:
    • OpenAPI Support
    • SOAP Scanner



To install them via the command line is even easier - just start ZAP with the command line flags: “-addoninstall soap -addoninstall openapi”
Eg:
./zap.sh -addoninstall soap -addoninstall openapi

Importing via the UI

These add-ons add the following items under the ‘Tools’ menu:
  • Import a WSDL file from local file system
  • Import a WSDL file from a URL
  • Import an OpenAPI definition from the local file system
  • Import an OpenAPI definition from a URL

These menu items will open new dialogs which will allow you to specify the relevant file or URL to import.

When the definitions have been imported they will be shown in the Sites tree:



Importing via the API

Both add-ons support importing API definitions from URLs and local files.
You can try these out via the API Web UI:


Using the Java API client this can be done with calls like:

Map<String, String> map1 = new HashMap<>();
map1.put("file", "/home/user/openapi.json");
ApiResponse resp =
  api.callApi("openapi", "action", "importFile", map1);

Map<String, String> map2 = new HashMap<>();
map2.put("url", "https://localhost/openapi.json");
ApiResponse resp =
  api.callApi("openapi", "action", "importUrl", map2);

Map<String, String> map3 = new HashMap<>();
map3.put("file", "/home/user/soap.xml");
ApiResponse resp =
  api.callApi("soap", "action", "importFile", map3);

Map<String, String> map4 = new HashMap<>();
map4.put("url", "https://localhost/soap.xml");
ApiResponse resp =
  api.callApi("soap", "action", "importUrl", map4);

Using the Python API client this can be done with calls like:

print zap._request(
  zap.base + 'openapi/action/importFile/',
  {'file':'/home/user/openapi.json'})

print zap._request(
  zap.base + 'openapi/action/importUrl/',
  {'url':'https://localhost/openapi.json'})

print zap._request(
  zap.base + 'soap/action/importFile/',
  {'file':'/home/user/soap.xml'})

print zap._request(
  zap.base + 'soap/action/importUrl/',
  {'url':'https://localhost/soap.xml'})

Spidering

Both of the add-ons automatically detect the relevant API definition file while spidering and will explore the definitions as long as they are in scope.
This means that you can explore an API by spidering from the URL of the definition as long as it is on the same domain as the API.

Next steps

Once you have added your API to the Sites tree using any of the above options you can then use any of the other ZAP components on the API, including the active scanner and fuzzer.

Note that the SOAP Scanner add-on also adds 2 additional scan rules that specifically target SOAP and XML URLs:


Monday, 6 February 2017

Introducing the JxBrowser add-on for ZAP

As modern web applications are increasing their reliance on JavaScript, security tools that do not understand JavaScript will not be able to work effectively with them.  ZAP already has components like the Ajax Spider and DOM XSS scanner that work by launching browsers and controlling them via Selenium, and we are planning to make much more use of browsers in the future.

To that end we are planning on releasing a new ZAP add-on which will contain JxBrowser, a wrapper around Chromium. We want ZAP to work as effectively as possible out of the box, and to be as easy to automate as possible. Being able to package JxBrowser in an add-on gives us an up to date browser that we know will work without any other user actions. While the add-on does contain a bundled version of Chromium it will not install Chrome on your computer. Your existing browsers and browser preferences will not be affected.

Please note: While JxBrowser is a commercial closed source wrapper around Chromium, TeamDev has generously given us a permanent free license to allow us to redistribute JxBrowser with ZAP. Most importantly ZAP will stay completely free. Additionally we will not be including the JxBrowser with the ZAP ‘core’ release which we maintain. It will also be very easy to remove the JxBrowser add-on from ZAP via the command line, in the same way that any other add-ons can be removed:
./zap.sh -addonuninstall jxbrowser

Why are we doing this?
As this is the first time we will have done anything like this I wanted to explain why we are packaging a closed source product with ZAP, what the implications are and how you can contact us to discuss any concerns you may have.

Unfortunately it is difficult for us to know which browsers will work on any specific system. Both Firefox and Chrome could be present, but we can’t tell until we try to launch them. IE and/or Edge are only going to be available on Windows systems and although Safari is always likely to be available on Mac OS it will not work with Selenium until an extra plugin is installed. As part of this on going effort we have recently decided to package the WebDrivers for Firefox, Chrome and IE in ZAP add-ons so that you will not have to download them manually. We want to be able to default to JxBrowser while allowing you to choose to use any of the browsers as you see fit. We will also be able to do things like launch JxBrowser from within ZAP pre-configured to proxy via ZAP.

Are there any licensing restrictions for JxBrowser?
As a ZAP user you will be able to use the ZAP JxBrowser add-on for any purposes.
However if you change the ZAP source code and call the JxBrowser API from one of your own products then you will need to arrange a suitable licence with TeamDev.
It is also worth noting that while JxBrowser does not ‘phone home’ the internal Chromium functionality may call 3rd party services (spell checking, geolocation, etc) in a similar way to other browsers.

When will the add-on be available?
It's available now :) You can install it from within ZAP using the 'Manage Add-Ons' button on the ZAP toolbar. Make sure that you update any add-ons that are flagged as needing updates, in particular the Selenium and Ajax Spider ones.

If you do have any concerns about our bundling of JxBrowser then please join in the discussion on the ZAP User Group or contact me directly.