Web Services and API

I made this using my notes and research during pentests and self training. I also used the 🚨 FREE course on API Sec University by Corey Ball 🚨 🚨 I really recommend that you check it out if not already. 🚨 I also used the module Introduction to Web Services and APIs from Hackthebox Academy

What are Web Services

As described by the World Wide Web Consortium (W3C): Web services provide a standard means of interoperating between different software applications, running on a variety of platforms and/or frameworks. Web services are characterized by their great interoperability and extensibility, as well as their machine-processable descriptions thanks to the use of XML.

Web services enable applications to communicate with each other.The applications can be entirely different. Consider the following scenario:

  • One application written in Java is running on a Linux host and is using an Oracle database

  • Another application written in C++ is running on a Windows host and is using an SQL Server database

These two applications can communicate with each other over the internet with the help of web services.

Definition taken from HTB Academy

What is an API

An application programming interface (API) is a set of rules that enables data transmission between different software. The technical specification of each API dictates the data exchange.

Consider the following example: A piece of software needs to access information, such as ticket prices for specific dates. To obtain the required information, it will make a call to the API of another software (including how data/functionality must be returned). The other software will return any data/functionality requested.

The interface through which these two pieces of software exchanged data is what the API specifies.

Definition taken from HTB Academy

Web Services vs API

The terms web service and application programming interface (API) should not be used interchangeably in every case.

  • Web services are a type of application programming interface (API). The opposite is not always true!

  • Web services need a network to achieve their objective. APIs can achieve their goal even offline.

  • Web services rarely allow external developer access, and there are a lot of APIs that welcome external developer tinkering.

  • Web services usually utilize SOAP for security reasons. APIs can be found using different designs, such as XML-RPC, JSON-RPC, SOAP, and REST.

  • Web services usually utilize the XML format for data encoding. APIs can be found using different formats to store data, with the most popular being JavaScript Object Notation (JSON).

Definition taken from HTB Academy

Web Services Approaches and Technologies

Definitions taken from HTB Academy

XML-RPC

XML-RPC uses XML for encoding/decoding the remote procedure call (RPC) and the respective parameter(s). HTTP is usually the transport of choice.

They do not mention it in HTB Academy, but I think it is important to know that a Remote Procedure Call (RPC) is a protocol used for communication between different systems over a network. It allows a client application to invoke procedures or functions on a remote server as if they were local procedures.

JSON-RPC

JSON-RPC uses JSON to invoke functionality. HTTP is usually the transport of choice.

SOAP (Simple Object Access Protocol)

SOAP also uses XML but provides more functionalities than XML-RPC. SOAP defines both a header structure and a payload structure. The former identifies the actions that SOAP nodes are expected to take on the message, while the latter deals with the carried information. A Web Services Definition Language (WSDL) declaration is optional. WSDL specifies how a SOAP service can be used. Various lower-level protocols (HTTP included) can be the transport.

Anatomy of a SOAP Message

  • soap:Envelope: (Required block) Tag to differentiate SOAP from normal XML. This tag requires a namespace attribute.

  • soap:Header: (Optional block) Enables SOAP’s extensibility through SOAP modules.

  • soap:Body: (Required block) Contains the procedure, parameters, and data.

  • soap:Fault: (Optional block) Used within soap:Body for error messages upon a failed API call.

WS-BPEL (Web Services Business Process Execution Language)

WS-BPEL web services are essentially SOAP web services with more functionality for describing and invoking business processes. WS-BPEL web services heavily resemble SOAP services. For this reason, they will not be included in this module's scope.

RESTful (Representational State Transfer)

REST web services usually use XML or JSON. WSDL declarations are supported but uncommon. HTTP is the transport of choice, and HTTP verbs are used to access/change/delete resources and use data.

Similar API specifications/protocols exist, such as Remote Procedure Call (RPC), SOAP, REST, gRPC, GraphQL, etc.

Web Services Description Language (WSDL)

WSDL stands for Web Service Description Language. WSDL is an XML-based file exposed by web services that informs clients of the provided services/methods, including where they reside and the method-calling convention.

A web service's WSDL file should not always be accessible. Developers may not want to publicly expose a web service's WSDL file, or they may expose it through an uncommon location, following a security through obscurity approach. In the latter case, directory/parameter fuzzing may reveal the location and content of a WSDL file.

The Definition is the root element of all WSDL files. Inside the definition, the name of the web service is specified, all namespaces used across the WSDL document are declared, and all other service elements are defined.

The definition is followed by the Data Types to be used in the exchanged messages

Messages: Defines input and output operations that the web service supports. In other words, through the messages element, the messages to be exchanged, are defined and presented either as an entire document or as arguments to be mapped to a method invocation.

Operation: Defines the available SOAP actions alongside the encoding of each message.

Port Type: Encapsulates every possible input and output message into an operation. More specifically, it defines the web service, the available operations and the exchanged messages. Please note that in WSDL version 2.0, the interface element is tasked with defining the available operations and when it comes to messages the (data) types element handles defining them.

Binding: Binds the operation to a particular port type. Think of bindings as interfaces. A client will call the relevant port type and, using the details provided by the binding, will be able to access the operations bound to this port type. In other words, bindings provide web service access details, such as the message format, operations, messages, and interfaces (in the case of WSDL version 2.0).

Service: A client makes a call to the web service through the name of the service specified in the service tag. Through this element, the client identifies the location of the web service.

Make a lab for API pentest

  • Install a kali VM

    • Update it

sudo apt update -y
sudo apt upgrade -y
sudo apt dist-upgrade -y
  • Install Burp Suite (its best if you can have a pro version)

    • Get the latest jython jar and install and specify the path in burp

    • Install Autorize from the store

  • Install foxy proxy in firefox

  • Set up burp certificate

  • Set up MITMweb certificate

    • quit burpsuite if open

    • set foxyproxy on port 8080

    • get the certificate from mitm.it and install it

  • Install postman sudo wget https://dl.pstmn.io/download/latest/linux64 -O postman-linux-x64.tar.gz && sudo tar -xvzf postman-linux-x64.tar.gz -C /opt && sudo ln -s /opt/Postman/Postman /usr/bin/postman (check for the dl link of the latest version and adapt the previous command)

  • Install mitmproxy2swagger sudo pip3 install mitmproxy2swagger

  • Install git if necessary

  • Install Docker sudo apt-get install docker.io docker-compose

  • Install Go sudo apt install golang-go

  • Set up the JSON Web Token Toolkit

git clone https://github.com/ticarpi/jwt_tool
cd jwt_tool
python3 -m pip install termcolor cprint pycryptodomex requests
(Optional) Make an alias for jwt_tool.py
sudo chmod +x jwt_tool.py
sudo ln -s /opt/jwt_tool/jwt_tool.py /usr/bin/jwt_tool
  • Install Kiterunner (see more info below)

  • Install Arjun sudo git clone https://github.com/s0md3v/Arjun.git

  • Install Owasp Zap if necessary sudo apt install zaproxy update the Add-Ons

  • Once you have done all this you can deploy a vulnerable api on another vm and connect it with your attacking machine. You will be all ready to practice.

Install crAPI and vAPI

crAPI

mkdir ~/lab 
cd ~/lab
sudo curl -o docker-compose.yml https://raw.githubusercontent.com/OWASP/crAPI/main/deploy/docker/docker-compose.yml
sudo docker-compose pull
sudo docker-compose -f docker-compose.yml --compatibility up -d
  • It may take a little while

  • Access it locally http://127.0.0.1:8025/ or http://127.0.0.1:8888/

  • To stop crAPI sudo docker-compose stop

vAPI

cd ~/lab
sudo git clone https://github.com/roottusk/vapi.git
cd vapi
sudo docker-compose up -d

Note: make sure that your port 80 is free

API Reconnaissance

  • Partner APIs are intended to be used exclusively by partners of the provider. These might be harder to find if you are not a partner. Partner APIs may be documented, but documentation is often limited to the partner.

  • Private APIs are intended for use, privately, within an organization. These APIs are often documented less than partner APIS, if at all, and if any documentation exists it is even harder to find.

Things to look for

Obvious URL naming schemes

https://target-name.com/api/v1 
https://api.target-name.com/v1 
https://target-name.com/docs
https://dev.target-name.com/rest
https://target-name.com/api-docs

dir names

/api, /api/v1, /v1, /v2, /v3, /rest, /swagger, /swagger.json, /doc, /docs, /graphql, /graphiql, /altair, /playground, /api-docs

Subdomains

api.target-name.com
uat.target-name.com
dev.target-name.com
developer.target-name.com
test.target-name.com

Request and Response headers

  • Use of JSON and XML

HTTP Request and Response Headers containing "Content-Type: application/json, application/xml"

Also, watch for HTTP Responses that include statements like:
{"message": "Missing Authorization token"}

Information gathered using third-party resources

Passive reconnaissance

See OSINT chapter for more detailed explaination useful for passive recon in general

Google Dorking

Google Dorking QueryExpected results

inurl:"/wp-json/wp/v2/users"

Finds all publicly available WordPress API user directories.

intitle:"index.of" intext:"api.txt"

Finds publicly available API key files.

inurl:"/api/v1" intext:"index of /"

Finds potentially interesting API directories.

ext:php inurl:"api.php?action="

Finds all sites with a XenAPI SQL injection vulnerability. (This query was posted in 2016; four years later, there are currently 141,000 results.)

intitle:"index of" api_key OR "api key" OR apiKey -pool

This is one of my favorite queries. It lists potentially exposed API keys.

Git Dorking

  • Specify parameters: "filename:swagger.json" and "extension: .json"

  • Search GitHub for your target organization’s name paired with potentially sensitive types of information, such as “api key,” "api keys", "apikey", "authorization: Bearer", "access_token", "secret", or “token.”

TruffleHog

  • sudo docker run -it -v "$PWD:/pwd" trufflesecurity/trufflehog:latest github --org=target-name

API Directory

Shodan

Shodan QueriesPurpose

hostname:"targetname.com"

Using hostname will perform a basic Shodan search for your target’s domain name. This should be combined with the following queries to get results specific to your target.

"content-type: application/json"

APIs should have their content-type set to JSON or XML. This query will filter results that respond with JSON.

"content-type: application/xml"

This query will filter results that respond with XML.

"200 OK"

You can add "200 OK" to your search queries to get results that have had successful requests. However, if an API does not accept the format of Shodan’s request, it will likely issue a 300 or 400 response.

"wp-json"

This will search for web applications using the WordPress API.

The Wayback Machine

  • Allows you to check out historical changes to your target.

  • Look for retired endpoints that still exist = Zombie APIs

  • Finding and comparing historical snapshots of API documentation can simplify testing for Improper Assets Management.

Active Reconnaissance

Nmap

  • We can run the following scan for any http port found

    • nmap -sV --script=http-enum <target> -p 80,443,8000,8080

OWASP Amass

  • amass enum -list see which data sources are available for Amass (paid and free)

  • sudo curl https://raw.githubusercontent.com/OWASP/Amass/master/examples/config.ini >~/.config/amass/config.ini create a config file to add our API keys

  • Register for a free Censys account here

  • Once it is done and that the account is verified login on search here

  • Go in my account and get your API ID and Secret

  • Paste them in the config file with sudo nano ~/.config/amass/config.ini

  • amass enum -active -d target-name.com |grep api

  • amass intel -addr [target IP addresses] collect SSL certificates, search reverse Whois records, and find ASN IDs associated to the target

  • amass intel -d [target domain] –whois reverse whois on domain name

  • amass enum -passive -d [target domain] passively enumerate subdomains

  • amass enum -active -d [target domain] actively enumerate subdomains

  • amass enum -active -brute -w /usr/share/wordlists/API_superlist -d [target domain] -dir [directory name] brute force subdomains with a wordlist

Gobuster

  • gobuster dir -u target-name.com:8000 -w /home/hapihacker/api/wordlists/common_apis_160

Kiterunner for active recon

  • See the documentation below.

DevTools

  • The devtools in firefox has a filter this way we can search for terms like API or v1 etc.

  • We can use the copy as cURL function to import it in Postman we just need to select Import and click on Raw Text and then paste the cURL request.

Kiterunner

  • We can fetch the latest release here or follow the guide if we want to build the binary.

    • Large routes list in kite fetchable here

    • Small routes list in kite fetchablehere

  • If go is not installed sudo apt update && sudo apt install golang

sudo git clone  https://github.com/assetnote/kiterunner.git
cd kiterunner
sudo make build
sudo ln -s /opt/kiterunner/dist/kr /usr/bin/kr
  • Useful commands:

    • Basic scan: kr scan https://<ip>/ -w routes-large.kite

    • Pass the Bearer token (to avoid getting too many 403): sudo ./kr scan https://<IP>/ -w routes-small.kite -H 'Authorization: Bearer TOKEN-HERE'

    • We can specify in the url /api/vX if we have it but kiterunner can also find it for us with the basic scan. Alissa Knight's whitepaper explains this very well: "By default, when depth isn’t specified, Kiterunner will scan a target at 1 level deep, meaning if you specified a target of server.com, Kiterunner would scan server.com/api. At level 2, it would scan server.com/api/R4 and at level 3, it would scan server.com/api/R4/patient."

    • To send to Burp Suite append the command with: --proxy=http://127.0.0.1:8080 (this is done with replay not with scan)

    • If you need to hide some specific response code that are not relevant for your context you can use this flag --fail-status-codes 404 this will blacklist 404 response codes to specify more than one you can do 404,500 for example.

  • Other interesting commands:

    • kr brute <target> -w ~/api/wordlists/data/automated/nameofwordlist.txt brute option will allow us to use txt files as wordlist

    • Replay requests:

    kr kb replay "GET     414 [    183,    7,   8]
    ://192.168.50.35:8888/api/privatisations/count 0cf6841b1e7ac8badc6e237ab300a90ca873d571" -w
    ~/api/wordlists/data/kiterunner/routes-large.kite
    • Get other interesting wordlists kr wordlist list scan with one of these lists kr scan https://target -A=name-of-wordlist

Be careful in prod environment it is really fast and might dos the server. You can be less noisy with the following flags:

  • -x, --max-connection-per-host – maximum number of open connections we can have on a host. Governed by 1 goroutine each. To avoid DOS’ing a host, we recommend keeping this in a low realm of 5-10. Depending on latency to the target, this will yield on average between 1-5 requests per second per connection (200ms – 1000ms/req) to a host.

  • -j, --max-parallel-hosts – maximum number of hosts to scan at any given time. Governed by 1 goroutine supervisor for each

Endpoint Analysis

Reverse Engineering an API

If an API is not documented or the documentation is unavailable to you, then you will need to build out your own collection of requests.

Manually with Postman

  • It's better to sign up for a free account.

  • We can create a new personal workspace

  • Then we can create a new collection

  • Now we will use the capture request feature so we will need to have added the postman proxy in foxy proxy previously

  • We need to enable the proxy

  • Start capture

  • You can then move to the web browser and browse in your target interact with it and do as many things as you can think of. This will populate the requests in postman.

  • Once you have browsed enough you can stop the capture

  • Go back to your request and select the ones that seem to interact with an API.

  • You can organize you collection say you have a path looking like api/folder1/something and one like api/folder2/something you can make a folder named folder1 and one named folder2 and organize your endpoints this way.

Automatic Documentation

  • mitmweb

  • set up you proxy on 8080 on your browser with foxyproxy

  • mitmproxy will open in the browser

  • Use the target Application

  • You will see the requests in mitmproxy

  • When you are done browsing you can save requests you will get a file named "flows"

  • Transform the file in an Open API 3.0 YAML file sudo mitmproxy2swagger -i Downloads/flows -o spec.yml -p http://target -f flow

  • edit spec.yml and remove the "ignore:" to the ednpoints you do not want to ignore (here we want to kepp all the api endpoints)

  • We can then rerun mitmproxy2swagger with the --examples flag sudo mitmproxy2swagger -i ~/Downloads/flows -o spec.yml -p http://target -f flow --examples

  • We can go to Swagger Editor

  • And import our spec.yml file

  • And you have a clean swagger ready to go :)

  • We can also import our spec.yml file in postman as a new collection.

API Documentation

  • overview = the first section of API documentation. Generally found at the beginning of the doc, it will provide a high-level introduction to how to connect and use the API. In addition, it could contain information about authentication and rate-limiting.

API Documentation Conventions

ConventionExampleMeaning

: or {}

/user/:id /user/{id} /user/2727 /account/:username /account/{username} /account/scuttleph1sh

The colon or curly brackets are used by some APIs to indicate a path variable. In other words, “:id” represents the variable for an ID number and “{username}” represents the account username you are trying to access.

[]

/api/v1/user?find=[name]

Square brackets indicate that the input is optional.

||

"blue" || "green" || "red"

Double bars represent different possible values that can be used.

General tips with Documentation

  • Check purposes of the various endpoints

  • Check out the collection variables

Tips Postman

  • Make sure that the baseUrl Value matches up with the URL to your target

  • Update Authorization for the collection. How to add a valid token:

    • You will be able to access the tab "Authorization"

    • In type select the token type you need. In crapi we will use Bearer Token

Note: If you want to add variables this is going to be similar you will have to select the tab variables and add it. Do not forget to save :)

RESTler

  • According to the documentation RESTler is "RESTler, developed at Microsoft, is the world’s first stateful API fuzzer that automatically generates tests and automatic execution by first reading the OpenAPI specification in order to automatically find vulnerabilities in the API."

  • Fetch it here

  • "If you receive a nuget error NU1403 when building, a quick workaround is to clear your cache with this command: $ dotnet nuget locals all--clear"

  • Quite a pain to install on kali.

wfuzz

This tool will allow to enumerate methods for API which is really convenient. It is preinstalled on kali.

  • wfuzz -X METHOD -w /path/to/wordlist -u http://URL/FUZZ where you see FUZZ it will replace with words from the wordlist and as for METHOD you can use GET POST and any other http verb

  • if you add the option --hc you can hide specific codes like 404 for instance --hc 404,405

  • To pass a header in wfuzz (Say Authorization for instance) you can use -H "myheader: headervalue" you can use multiple header this way

  • Check out this doc for more infor on this tool

Convert Postman to OpenApi

  • You can convert them using Postman to OpenApi

  • sudo apt install npm Install nmp if you do not have it

  • sudo npm i postman-to-openapi -g install postman to openapi cli mode

  • p2o myfile.postman_collection.json -f myfile.yml convert your first file

About response codes

  • Do not neglect 400 codes, they can help you identify actual endpoint. For example 404 is not found but 401 is unauthorized.

  • However be also aware that:

    • Sometimes you will get unauthorized on every endpoint meaning that this theory won't work ;)

    • Sometimes waf will drop your requests or will give you a 200 with an error page

Postman

Setting up baseurl

Bearer Token

If you use a bearer token and do not want to set up authentication, you can just add a current token in the Authozization tab or the collection's root. Important: Do not forget to Save your changes because it wont work if you don't After this you just have to set the other to inherit auth from parent and you should be good to go.

Scanning API

Note that this step should be done AFTER you have discovered an API and used it as it was intended. You will then avoid being blocked by a WAF

  • The goal here is to guide and focus the testing. Because, as Corey mentions scans are rarely 100% accurate

  • In the case of API, it will be helpful to find Security Misconfiguration (see details on this vunerability below)

Scan with OWASP ZAP

  • You can specify the target URL. But as Corey mentions in his course, the result are improved if we use the target's API Specification file. This file is the documentation we generated when we explored the API and used it as intended or the one that your customer might have provided to you when kicking off the pentest.

  • Now ZAP GUI should be populated with the endpoint.

  • Enable the HUD, Lauch browser and browse around as much as you can.

  • Then launch another active scan by switching the Attack mode to "On"

If the HUD does not show here are a few things you should check:

  • Check that all is up to date in ZAP Help > Check for updates

Scan with Burp (requires pro account)

  • You can follow the same methodology we did for ZAP, scan unauthenticated and then authenticate browse around and rescan.

  • You might need to edit the yaml or json file to add the actual host (depending on how you got the documentation)

Special thanks to my friend Lisandre for pointing me to the Extension

API vulnerabilities

API1:2019 Broken Object Level Authorization

API3:2019 Excessive Data Exposure

If you get a response and:

  • It includes more information than what was requested

  • It has sensitive Information that can be leveraged in more complex attacks

  • For example in Crapi

  • Here we get usernames emails etc.

API3:2019 Excessive Data Exposure - OWASP

API7:2019 - Security Misconfiguration

Security misconfiguration includes missing system patches, unnecessary features enabled, lack of secure transit encryption, weak security headers, verbose error messages, and Cross-Origin Resource Sharing (CORS) policy misconfigurations.

  • It can be found manually of course, but if you want to speed up the process automatic scans are efficient to detect these.

API7:2019 Security Misconfiguration - OWASP

API9:2019 Improper Assets Management

API Attacks or specific vulnerabilities tests

Classic Authentication Attacks

For API pentest we can try these classic attacks such as password spraying, bruteforcing or password reset bruteforce.

Password Brute-Force Attacks

  • The request will be sent to an API endpoint

  • Payload often in JSON

  • Might require base64 encoding

  • Tip: generate passwords specific to your target

  • Tools: Burp Suite Intruder, WFuzz, etc.

  • Wordlist: rockyou or you can find plenty of wordlist online such as this repo: SecList

WFuzz on API

  • -H Header option

  • --hc, --hl, --hw, --hh Hide responses with the specified code/lines/words/chars

  • -d POST body requests = Use post data (ex: "id=FUZZ&catalogue=1") allows you to fuzz content that is sent in the body of a POST request

  • Example with crAPI wfuzz -d '{"email":"a@email.com","password":"FUZZ"}' -H 'Content-Type: application/json' -z file,/usr/share/wordlists/rockyou.txt -u http://127.0.0.1:8888/identity/api/auth/login --hc 405

  • status codes in the 200s and 300s should be good indicators that you have successfully brute-forced credentials

Password spraying

Many security controls could prevent you from successfully brute-forcing an API’s authentication. A technique called password spraying can evade many of these controls by combining a long list of users with a short list of targeted passwords. The real key to password spraying is to maximize your user list. The more usernames you include, the higher your odds of compromising a user account with a bad password. Build a user list during your reconnaissance efforts or by discovering excessive data exposure vulnerabilities.

Pull email with grep

I am a huge fan of grep (but not of regex -_-'). And in his course Corey shares a great grep command to use on a json response you got from excessive data exposure and that you saved in a file with postman.

  • grep -oe "[a-zA-Z0-9._]\+@[a-zA-Z]\+.[a-zA-Z]\+" response.json

  • You can then use a command like sort -u to get rid of duplicate emails

How to proceed with burp intruder

In proxy Positions we add our var in email and password values and we select the "Cluster Bomb" attack type. In the payload we use simple list and load user and passwords list for each payload set. Once done we just need to analyze results. In case of base64 encoding we juste need to use the Payload Processinf rule "Base64-encode"

Credential Stuffing with Intruder

  • If you have a CSV file with creds (like the one provided for vAPI) the more convenient way to pass it in the intruder is to proceed as follow

  • Set up payload set 1 as follow, load the csv file and make a payload processing with a match on ,.* and leave the replace empty. You might not need the encode char options depending on your target

  • Set up Payload set 2 as follow, load the csv file and make a payload processing with a match on .*, and leave the replace empty. You might not need the encode char options depending on your target.

  • Launch the attack and analyze the results (you can play with Status and length for the analysis). In our example we can see the 200 codes that indicates a successful login.

API Token Attacks

Analyze tokens with Burp Sequencer

  • Right click in the request and send it to the sequencer

  • In configuration Define the start and the end of the token

  • Start live capture

  • Analyzing the results will allow you to know if the tokens are predictable or not.

  • Corey referenced this repo of bad tokens if we want to see what the sequencer results look like with bad tokens

JWT Attacks

JWTs consist of three parts, all of which are base64 encoded and separated by periods: the header, payload, and signature. JWT.io is a free web JWT debugger that you can use to check out these tokens. You can spot a JWT because they consist of three periods and begin with "ey". They begin with "ey" because that is what happens when you base64 encode a curly bracket followed by a quote, which is the way that a decoded JWT always begins.

  • If you’ve captured another user’s JWT or maybe discovered a leaked token during reconnaissance, you can try sending it to the provider and pass it off as your own. There is a chance that the leaked token has not yet expired and can be passed off as your own.

For this part in addition to Corey's course I really recommend the following video by Farah Hawa

ATTACKING JWT FOR BEGINNERS! - Farah Hawa

JWT Tool

  • With this we can: analyze JWTs, scan for weaknesses, forge tokens, and brute-force signature secrets.

  • JWT options

    • -h to show more verbose help options

    • -t to specify the target URL

    • -M to specify the scan mode

    • pb to perform a playbook audit (default tests)

    • at to perform all tests

    • -rc to add request cookies

    • -rh to add request headers

    • -rc to add request cookies

    • -pd to add POST data

  • jwt_tool <token-value> Will give details on the token. (you migh to run it 2 times because on the first run it needs to generate a config file along other utility files)

  • jwt_tool -t http://target-name.com/ -rh "Authorization: Bearer JWT_Token" -M pb scan for common vulnerabilities

The None Attack

  • When a JWT is using "none" as its algorithm.

  • Easy win

  • After decoding the token, you should be able to clearly see the header, payload, and signature.

  • Change username to get more rights

  • Once you’ve edited the payload, use Burp Suite’s Decoder to encode the payload with base64; then insert it into the JWT. Importantly, since the algorithm is set to "none", any signature that was present can be removed. In other words, you can remove everything following the third period in the JWT. Send the JWT to the provider in a request and check whether you’ve gained unauthorized access to the API.

The Algorithm Switch Attack

There is a chance the API provider isn’t checking the JWTs properly. If this is the case, we may be able to trick a provider into accepting a JWT with an altered algorithm. One of the first things you should attempt is sending a JWT without including the signature. This can be done by erasing the signature altogether and leaving the last period in place, like this: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJnYWJ0ZXN0MUB0ZXN0LmNvbSIsInJvbGUiOiJ1c2VyIiwiaWF0IjoxNjczNDUzOTMwLCJleHAiOjE2NzQwNTg3MzB9. If this isn’t successful, attempt to alter the algorithm header field to "none". Decode the JWT, update the "alg" value to "none", base64-encode the header, and send it to the provider. To simplify this process, you can also use the jwt_tool to quickly create a token with the algorithm switched to none.

jwt_tool eyJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJnYWJ0ZXN0MUB0ZXN0LmNvbSIsInJvbGUiOiJ1c2VyIiwiaWF0IjoxNjczNDUzOTMwLCJleHAiOjE2NzQwNTg3MzB9.c2jSKVKV2Fa-0Z8RmmquEMBggfaWB7kcb7VcqiuN4DYHNXCHYPtd-K93C4tcL8weg7M408hW4-XlLA6xBFxdJ41Io24CZRHkzrSTVIrdMxDPB8tTQUR8Ib61gMAZUeWb7IGHLgpyLCVBDyqy6q1cs6EFNcYVV69N7zakTTRUBKvNnvA62F0Zz7Rf_9YONP9U3nvn_qQJBQo1P3OoWsasuz1KIog-Z0mkRl-d53mJtXbyFiJDzgImzD-nKgQ4sKza2nqSguo1qqfLuf029Kmb7N2nxen8tHdkRht6TR5tZrXMxt047tQ8NCPj6ql-kEcBKwbHc4z4IpeSCtzkSR9wYw -X a
  • If provider accepts multiple algorithms, we can alter the algorithm. Example provided by Corey: For example, if the provider uses RS256 but doesn’t limit the acceptable algorithm values, we could alter the algorithm to HS256. This is useful, as RS256 is an asymmetric encryption scheme, meaning we need both the provider’s private key and a public key in order to accurately hash the JWT signature. Meanwhile, HS256 is symmetric encryption, so only one key is used for both the signature and verification of the token. If you can discover and obtain the provider’s RS256 public key then switch the algorithm from RS256 to HS256, there is a chance you may be able to leverage the RS256 public key as the HS256 key.

  • jwt_tool TOKEN -X k -pk public-key.pem to leverage this attack with jwt_tool (we will get a new token)

JWT Crack Attack

  • We try to crack the secret used for the JWT signature hash. We will try this on crAPI

  • crunch 5 5 -o crAPIpw.txt create a list of all possible character combinations to use against crAPI. It will create a wordlist of 5 chars

  • With the key we can create any token we want

Crunch is a really great tool. See more use cases here

Other attacks or tools

Exploiting API Authorization

Broken Object Level Authorization (BOLA)

  • Access to resources that do not belong to you

  • Ingredients for successful exploitation

    • Resource ID: a resource identifier will be the value used to specify a unique resource. This could be as simple as a number, but will often be more complicated.

    • Requests that access resources. In order to test if you can access another user's resource, you will need to know the requests that are necessary to obtain resources that your account should not be authorized to access.

    • Missing or flawed access controls. In order to exploit this weakness, the API provider must not have access controls in place. This may seem obvious, but just because resource IDs are predictable, does not mean there is an authorization vulnerability present.

  • Finding Resource IDs and Requests

    • Identify structure and patterns

GET /api/resource/1
GET /user/account/find?user_id=15
POST /company/account/Apple/balance
POST /admin/pwreset/account/90
  • Ideas of requests and patterns

  • Questions to ask yourself:

    • What can you do with this app?

    • Do you get your own profile?

    • Can you upload files?

    • Do you have an account balance?

    • Is there any part of the app that has data specific to your account?

  • Testing strategy (A-B Testing)

    1. Create a UserA account.

    2. Use the API and discover requests that involve resource IDs as UserA.

    3. Document requests that include resource IDs and should require authorization.

    4. Create a UserB account.

    5. Obtaining a valid UserB token and attempt to access UserA's resources.

BOLA Example in crAPI

  • Example of exploitation. In crAPI, we create 2 users and use the token of user B to access the location of the vehicule of user A

BOLA Example in vAPI

  • In vAPI we can access another user info just by specifying the id in the endpoint.

Broken Function Level Authorization

  • Perform unauthorized actions

  • Ingredients for successful attack

    • Resource ID: a resource identifier will be the value used to specify a unique resource.

    • Requests that perform authorized actions. In order to test if you can access another update, delete, or otherwise alter other the resources of other users.

    • Missing or flawed access controls. In order to exploit this weakness, the API provider must not have access controls in place.

  • Hunting for functional requests: POST, PUT, DELETE, and potentially GET with parameters.

Important reminder by Corey: Please take note: When successful, BFLA attacks can alter the data of other users. This means that accounts and documents that are important to the organization you are testing could be on the line. DO NOT brute force BFLA attacks, instead, use your secondary account to safely attack your own resources. Deleting other users' resources in a production environment will likely be a violation of most rules of engagement for bug bounty programs and penetration tests

  • In crAPI, we can illustrate this attack by updating video the name of the video of user a with the token of user B. This is going to be only lateral

  • Here we make the legitimate request of user A updating their video

  • Here is the attack of the request with user B token to update the video of user A

  • To make this attack and elevate our privilege, we can try to delete a video.

  • When modifying to DELETE we get this message

{"message":"This is an admin function. Try to access the admin API","status":403}
  • In the endpoint /identity/api/v2/user/videos/33 it specifies user what if we change this to admin. It works. We elevated our privilege and where able to make admin actions

Corey mentions that the Authorization vulnerabilities continue to be the most common API vulnerabilities.

Testing for Improper Assets Management

  • Discovering unsupported and non-production versions of an API.

  • Examples of path

api.target.com/v3
/api/v2/accounts
/api/v3/accounts
/v2/accounts
  • Examples of headers

Accept: version=2.0
Accept api-version=3
  • Example of query parameter or request body

/api/accounts?ver=2
POST /api/accounts

{
"ver":1.0,
"user":"hapihacker"
}
  • Non-production versions of an API include any version of the API that was not meant for end-user consumption

api.test.target.com
api.uat.target.com
beta.api.com
/api/private
/api/partner
/api/test
  • Once we have discovered an unsupported version of the API, we will test for additional weaknesses.

How to find

You can discover mass assignment vulnerabilities by finding interesting parameters in API documentation and then adding those parameters to requests. Look for parameters involved in user account properties, critical functions, and administrative actions. Intercepting API requests and responses could also reveal parameters worthy of testing. Additionally, you can guess parameters or fuzz them in API requests that accept user input. I recommend seeking out registration processes that allow you to create and/or edit account variables.

How to test

  • Test from both unauthenticated and authenticated perspectives.

  • We can the collection runner (it will test requests across the entire collection) in Postman multiple times along with an environmen variable {{ver}} to specify the version we want to look for.

  • To use the var we first need to use the find an Replace function (on bottom of the Postman window), find all the v2 an replace them with {{ver}}

  • Then we just need to add the variable in our variables update its value to another version (v1 for example), run the collection runner and analyze the results.

Mass Assignment

  • An attacker is able to overwrite object properties that they should not be able to.

  • Ingredients

    • Requests that acceot user input and alter values available to the user

    • Missing security controls

How to find Mass Assignment vulnerabilities

  • Find interesting parameters in API documentation, add them to requests.

    • Look for parameters involved in user account properties, critical functions, and administrative actions.

  • Use AOI as intended to study parameters used by API provider

    • Understand names and spelling conventions

  • Fuzz parameter values in requests

  • Check the target's account registration process if present

Example with crAPI

  • If we do not have admin doc we could try these

"isadmin": true,
"isadmin":"true",
"admin": 1,
"admin": true, 
  • Or we could fuzz by sending the request to the intruder and setting the vars on the parameter we wish to add

Param Miner extension on Burp

  • Install it from the extender

  • And then in the window that pops up we select the options we need

Other Mass Assignment Vectors

  • More than trying to become adnin

    • Gain unauthorized access to other organizations

    • Example

      POST /api/v1/register
      --snip--
      {
      "username":"hAPI_hacker",
      "email":"hapi@hacker.com",
      "org": "§CompanyA§",
      "password":"Password1!"
      }
  • API functions other than registration can be vulnerable

Example with vAPI

  • In vAPI the documentation specifies that api6 is dedicated for testing mass assignment

  • Let's first use /vapi/api6/user/me to see the parameters we get. Remember here we need to encode our creds in the format user:pass in base64 and pass them in the Authorization-Token header.

  • We have a credit parameter let's try to add ourselves 800 credits with a post request on this endpoint /vapi/api6/user.

  • It seems to work, let's check this with the other endpoint

  • It works and we even get a flag

Server-Side Request Forgery (SSRF)

  • An application retrieves remote resources without validating user input

  • An attacker can supply their own input, in the form of a URL, to control the remote resources that are retrieved by the targeted server. Possible consequences

    • Gain access to sensitive data

    • Completely compromise the target host

  • Growing threats on API

Types of SSRF

In-Band SSRF

  • For an In-Band SSRF, a URL is specified as an attack. The request is sent and the content of your supplied URL is displayed back to you in a response. Once you have discovered an In-Band SSRF vulnerability you could leverage control over the URL to scan the internal network environment, gather sensitive information from the localhost, or attempt to perform a remote code execution attack.

  • Example

    • Intercepted Request

    POST api/v1/store/products
    headers…
    {
    "inventory":"http://store.com/api/v3/inventory/item/12345"
    }
    • Attack:

    POST api/v1/store/products
    headers…
    {
    "inventory":"§http://localhost/secrets§"
     }
    • Response

    HTTP/1.1 200 OK
    headers...
    {
    "secret_token":"crapi-admin"
    }

Blind SSRF

  • Blind (or Out of Band) SSRF takes place when a vulnerable server performs a request from user input but does not send a response back to the user indicating a successful attack. The app does not provide an unusual response to the user, but the server does make the request to the URL specified by the attacker. In this case, to know if the request was made you will need to have some control over the web server that is specified in the attack. In this case, the response is returned and we do not have any indication that the server is vulnerable. Instead of http://localhost/secrets, we will need to provide the URL to a web server that will let us see if a request is actually made. Burp Suite Pro has a great tool called Burp Suite Collaborator. Collaborator can be leveraged to set up a web server that will provide us with the details of any requests that are made to our random URL. You can also use one of these other free sites:

  • canarytokens

    In case of pentest you might want to setup your own web server instead of using public tools. You can also deploy your own private Burp Suite Collaborator server. See how to do this here. In a CTF context a simple python3 -m http.server 80 usually does the trick too.

  • Example

    • Intercepted Request

    POST api/v1/store/products
    headers…
    {
    "inventory":"http://store.com/api/v3/inventory/item/12345"
    }
    • Attack:

    POST api/v1/store/products
    headers…
    {
    "inventory":"§https://webhook.site/306b30f8-2c9e-4e5d-934d-48426d03f5c0§"
    } 
    • Response:

    HTTP/1.1 200 OK
    headers...
    {}

Ingredients for SSRF

Look for requests that have any of the following:

  • Include full URLs in the POST body or parameters

  • Include URL paths (or partial URLs) in the POST body or parameters

  • Headers that include URLs like Referer

  • Allows for user input that may result in a server retrieving resources

SSRF Example with crAPI

  • It is worth fuzzing the parameter where you will put the url to request using the intruder to check what is working. We can use PayloadAllTheThings to find Payloads to try out.

  • In crAPI's case the endpoint /workshop/api/merchant/contact_mechanic is worth fuzzing.

  • We can see on collaborator webhook and my webserver that these request worked

  • From the results we can see that we can not query localhost but we can query the rest.

SSRF Example with vAPI

  • In vapi if we can try this endpoint /vapi/serversurfer

Injection Attacks

  • Discover by fuzzing all potentials inputs

    • Headers

    • Query string parameters

    • Parameters in POST/PUT requests

Interesting note by Corey: The more you know about the API, the more you can focus your attacks and increase your chance of finding a vulnerability. This is where your reconnaissance efforts will really pay off. If you know what database the application uses, what operating system is running on the web server, or the programming language in which the app was written, you’ll be able to submit targeted payloads aimed at detecting vulnerabilities in those particular technologies.

  • When reviewing API documentation, if the API is expecting a certain type of input (number, string, boolean value) send:

    • A very large number

    • A very large string

    • A negative number

    • A string (instead of a number or boolean value)

    • Random characters

    • Boolean values

    • Meta characters

SQL Injection Metacharacters

  • SQL Metacharacters are characters that SQL treats as functions rather than data. For example, -- is a metacharacter that tells the SQL interpreter to ignore the following input because it is a comment.

'
''
;%00
--
-- -
""
;
' OR '1
' OR 1 -- -
" OR "" = "
" OR 1 = 1 -- -
' OR '' = '
OR 1=1

NoSQL Injection

  • APIs commonly use NoSQL databases due to how well they scale with the architecture designs common among APIs. Also, NoSQL injection techniques aren’t as well-known as their structured counterparts. Due to this one small fact, you might be more likely to find NoSQL injections.

  • Common NoSQL metacharacters

$gt 
{"$gt":""}
{"$gt":-1}
$ne
{"$ne":""}
{"$ne":-1}
 $nin
{"$nin":1}
{"$nin":[1]}
{"$where":  "sleep(1000)"}

OS Injection

  • Command separators to pair multiple commands

|
||
&
&&
'
"
;
'"
  • Common OS commands

Operating systemCommand

Windows

ipconfig shows the network configuration.

dir prints the contents of a directory.

ver prints the operating system and version.

whoami prints the current user.

*nix (Linux and Unix)

ifconfig shows the network configuration.

ls prints the contents of a directory.

pwd prints the current working directory.

whoami prints the current user.

  • Find more on linux here

  • Find more on windows here

Fuzzing an entire API collection with Postman

  • The collection runner in postman is really convenient for fuzzing an entire API collection

  • We can duplicate a collection, select multiple interesting requests, set fuzz variables, proceed with various payloads and analyze the results.

  • When a request gives interesting results, we can send it to the intruder or WFuzz

  • If you want to fuzz an interesting request with wfuzz take the request from burp by saving it in a file with "copy to file" option and use the file and wfuzz --help to write your wfuzz command

Injection Example with crAPI

  • We follow the process described by Corey with postman.

  • We send the interesting request to the intruder, in crAPI it is this endpoint /community/api/v2/coupon/validate-coupon. During the analysis with postman we got interesting results with noSQL injection payloads

  • We set the Payloads type (I used the list given by Corey you can find it above)

Here is the command that Corey uses to fuzz this request with WFuzz wfuzz -z file,usr/share/wordlists/nosqli -H "Authorization: Bearer TOKEN" -H "Content-Type: application/json" -d "{\"coupon_code\":FUZZ} http://crapi.apisec.ai/community/api/v2/coupon/validate-coupon He then uses --sc 200 to filter out the results. To better analyze results he recommends proxying wfuzz traffic through burpsuite -p localhost:8080

Injection example with vAPI

  • In the documentation we can see that the request to test for injection is this endpoint in api8 /vapi/api8/user/login

  • Let's fuzz this specific request with burp

Evasion and combining techniques

  • A really interesting chapter in APISec Uni on evasion techniques.

Evasive Maneuvers

String terminators

Null bytes and other combinations of symbols are often interpreted as string terminators used to end a string. If these symbols are not filtered out, they could terminate the API security control filters that may be in place. When you are able to successfully send a null byte it is interpreted by many back-end programming languages as a signifier to stop processing. If the null byte is processed by a back-end program that validates user input, then the security control could be bypassed because it stops processing the following input.

%00
0x00
//
;
%
!
?
[]
%5B%5D
%09
%0a
%0b
%0c
%0e

String terminators can be placed in different parts of the request, like the path or POST body, to attempt to bypass any restrictions in place. For example, in the following injection attack to the user profile endpoint, the null bytes entered into the payload could bypass filtering

POST /api/v1/user/profile/update
[…]
{
“uname”: “hapihacker”
“pass”: "%00'OR 1=1"
}

In this example, input validation measures might be bypassed due to the null byte placed before the SQL injection attempt.

Case Switching

  • Examples

    • Original request

      POST /api/myprofile 
      […] 
      {uid=§0001§} 
    • Modified requests with case switching

      POST /api/myProfile 
      POST /api/MyProfile 
      POST /aPi/MypRoFiLe 

Encoding Payloads

To take your WAF-bypassing attempts to the next level, try encoding payloads. Encoded payloads can often trick WAFs while still being processed by the target application or database. Even if the WAF or an input-validation measure blocks certain characters or strings, they might miss encoded versions of those characters. Alternatively, you could also try double encoding your attacks.

Imagine that the API provider has a control in place that provides one round of decoding to requests that are received.

URL Encoded Payload: %27%20%4f%52%20%31%3d%31%3b

API Provider URL Decoder: ' OR 1=1;

WAF Rules detect a fairly obvious SQL Injection attack and block the payload.

Double URL Encoded Payload: %25%32%37%25%32%30%25%34%66%25%35%32%25%32%30%25%33%31%25%33%64%25%33%31%25%33%62

API Provider URL Decoder: %27%20%4f%52%20%31%3d%31%3b

This could perhaps pass by a bad WAF rule to later be decoded and interpreted.

Payload processing with burp

  • When we can successfully bypass a waf, we can use the payload processing option in Burp Intruder when fuzzing

  • Intruder > Payloads > Payload Processing to add the rule you need.

  • Test by starting an attack and update if necessary

Evasion with WFuzz

  • See here the advanced secion of the WFuzz docs to find the payload processing options.

  • wfuzz -e encoders to see a list of all Wfuzz encoders

CategoryNameSummary

hashes

base64

Encodes the given string using base64

url

urlencode

Replaces special characters in strings using the %xx escape. Letters, digits, and the characters '_.-' are never quoted.

default

random_upper

Replaces random characters in strings with capitals letters

hashes

md5

Applies a MD5 hash to the given string

default

none

Returns all characters without changes

default

hexlify

Converts every byte of data to its corresponding two-digit hex representation

To use an encoder, add a comma to the payload and specify its name.

  • wfuzz -z file,wordlist/api/common.txt,base64 http://url-of-your-target/FUZZ

  • wfuzz -z list,TEST,base64-md5-none To have a payload processed multiple encoders specify them with a hyphen

Combining Techniques

  • Combining multiple techniques together is what will help you find critical vulnerabilities, increase the impact of your bounties, and more closely emulate advanced attackers.

  • Once you have found a small crack in the security of an API then you may have found a launchpad for the rest of your attack.

  • Excessive Data Exposure and Improper Assets Management are both API weaknesses that stand out above the others to combine in order to escalate the impact of your attacks.

  • Directory traversal fuzzing attack along with the file upload to manipulate where the file is stored

  • Use Broken Function Level Authorization (BFLA) or Mass Assignment weaknesses to search for Improper Assets Management, injection weaknesses, or Broken Object Level Authorization weaknesses.

  • When you have discovered mass assignment fuzz the parameter for other weaknesses.

  • If you are able to update to perform a BFLA attack and update another user’s profile information perhaps you could combine this with an XSS attack.

  • Combine Injection attacks with most other vulnerabilities.

  • If you find a BOLA vulnerability that provides you with other users’ bank account information check to see if you are also able to exploit a BFLA vulnerability to transfer funds.

  • If you are up against API security controls like a WAF or rate-limiting, make sure to apply evasion techniques to your attacks. Encode your attacks and attempt to find ways to bypass these security measures. If you have exhausted your testing efforts, return to fuzzing wide with encoded and obfuscated attacks.

Swagger-ui

If you want to replace postman for some reason, you can use swaggerHUB. It is really straightforward.

  • Launch Apache in your machine sudo systemctl start apache2.service (use status instead of start to check if it is running first)

  • Download the zip file from here

  • And copy and paste the dist folder in your /var/www/html/ folder.

  • You can then browse as you wish http://127.0.0.1/dist/index.html

  • This way you can modify your swagger and send your request through the browser.

  • If you have a yaml file you can convert it in json.

sudo apt update
sudo apt install yq
# convert yaml to json
yq -p yaml -o json input.yml
# convert json to yaml
yq -p json -o yaml input.json
# if jq or yq is pain you can also use perl or python
## convert yaml to json in perl
ruby -ryaml -rjson -e 'puts JSON.pretty_generate(YAML.load(ARGF))' < file.yml > file.json
## convert yaml to json in python
python -c 'import sys, yaml, json; json.dump(yaml.load(sys.stdin), sys.stdout, indent=4)' < file.yaml > file.json
  • Then you just need to specify the url to your json in the search bar like this and click explore. You url to your json would look like http://127.0.0.1/dist/myswagger.json considering you've put your swagger in the /dist folder.

  • If you want to see lively the request made to your apache server you can run as root (sudo su) tail -f apache2/access.log or tail -f /var/www/log/apache2/other_vhosts.access.log depending on your setup

  • If you want your server to have a hostname you need to edit your /etc/hosts file like this

127.0.1.1       mycustomhostname

Misc tips

Misc

  • With an API we need to check as many endpoints as possible sometimes an endpoint will be protected but another will not have the same protection.

  • A great methodology to use is the one made by David Sopas - MindAPI

  • Do not forget also to try to enumerate id if for instance you have http://api/user/ try to put 1 in the end

  • When interactig with json in burpsuite repeater if we find an endpoint we have to add the content type like this in the headers Content-Type: application/json

  • Do not hesitate to tamper with responses also (not only requests)

Sharepoint API

If you see an endpoint looking similar to this SP.Web.SOMETHING it means that your target is using a Sharepoint API.

There are a few things that you could try with this.

Note: have a look at the resources provided in the end about sharepoint APIs they can give you plenty of ideas for your pentest

Oasdiff - Check 2 version of the same API

Resources

Writeups

I think we learn a lot when reading examples of exploitations. Here is a list of writeups with API.

HTB write ups with API

Vulnerable API

Tools

Postman
Kiterunner
API Recon with Kiterunner - Hacker Toolbox - InsiderPHD

Courses, papers or articles

Resources and tools about Sharepoint APIs

Standards and references

Wordlists

Practice and labs

List of resources

Last updated