blog.smigiel.dev

Nov 20, 2020

Scanning Network

Mapping a Network

Sometimes, instead of receiving full information about target machines, penetration tester gets just address block. Later on, pentester needs to discover what kind of hosts with what kind of configuration, exist in particular network.

There are several different ways to reveal network configuration.

PING sweeping

Ping works by sending one or more special ICMP packets (type 8 - echo request) to a host. If the host replies with ICMP echo reply packets, it means that host is alive. RFC792 describes protocol used to carry diagnostic messages. ICMP is a part of the Internet Protocol.

fping

fping is a Linux tool, improved version of standard ping. It can be run against IP range:

fping -a -g <ip_range> 2>/dev/null # -a -- show alive; -g -- generate list, redirect to /dev/null to surpress noise.

nmap

Recommended tool for penetration testing is nmap. It is very powerful tool, which allows for detection hosts, its systems and more. To attempt port sweeping (ping scan), one can use -sn option:

nmap -sn <ip_range>

OS Fingerprinting

After finishing nmap run, we end up having list of live hosts, responding to ping. Next, we need to understand what kind of operating system is used by a host. Based on differences in network stack implementation of the various operating systems, automated programs can analyze response, and recognize OS version, creating host signatures.

During a penetration test, person needs to perform reconnaissance step on every network node:

  • routers
  • firewalls
  • hosts
  • servers
  • printers
  • etc

The goal is to create table of nodes, with appropriate information about systems.

Offline fingerprinting (p0f)

For offline fingerprinting, one can use p0f tool to analyze dump of network traffic.

Online fingerprinting (nmap)

In a case of online, to go tool is again nmap. To perform OS fingerprinting with it, you have to use -O option and specify target. In a case of known targets, you can add -Pn to skip rediscovering them again.

nmap -Pn -O <targets>

This option can be additionally fine-tuned, to prevent from too aggressive way of discovering hosts:

OS DETECTION:
  -O: Enable OS detection
  --osscan-limit: Limit OS detection to promising targets
  --osscan-guess: Guess OS more aggressively

Port Scanning

After recognizing nodes and detecting operating systems, we can continue with port scanning. It allows to discover the daemons and services running on those nodes.

Port scanning is a process used to determine what TCP and UDP ports are open on target hosts. It, also usually helps detect which software and version is listening on a specific port.

TCP Three-Way Handshake

Default usage:

nmap -sT <target>

Description of process.

Client --- SYN ---> Server
Client <- SYN+ACK - Server
Client --- ACK ---> Server

Client wants to connect to a server, it first sends a packet with the SYN flag enabled. The server then responds by sending a packet with both SYN and ACK flags enabled. Finally, the client replies back by sending a packet with the ACK flag enabled and the actual data transmission can start.

In a case of closed port, it looks slightly differently.

Client --- SYN ---> Server
Client <- RST+ACK - Server

The server will reply with a packet that has the reset RST and ACK flags set. This behavior tells the client that the port is closed.

The simplest way to perform a port scan is trying to connect to every port.

  • if the scanner receives a RST packet, then the port is closed.
  • if the scanner can complete 3-way handshake, then the port is open. After connecting, the scanner sends an RST packet to the target host to abruptly close the connection.

The culprit here is, that every TCP connect is recorded in the daemon logs. Even scans, which often should be stealth. That is because, from the application point of view, the scan probe is legitimate connection. It allows system administrators to easily detect the scan.

TCP-SYN scan

Default usage:

nmap -sS <targets>

To mitigate possibility of being detected (in the case of 3-way handshake), there is more stealthy solution, called TCP-SYN scan. During a SYN scan, the scanner does not perform a full handshake. It just sends a SYN packet and analyzes the response coming from the target machine.

  • if it receives a RST packet, then port is closed.
  • if it receives an ACK packet, then port is open. After marking the port as open, the scanner sends a RST packet to the target host to stop the handshake.

As there is no full connection to the destination daemon, a SYN scan cannot be detected by looking at daemon logs. It is important to remember, that even TCP-SYN scan can be detected with well-configured IDS (Intrusion Detection System).

Version detection scan

Default usage:

namp -sV <targets>

This scan is recorded in logs, however it is very useful. It mixes a TCP connect scan with some probes, which are used to detect what application is listening on a particular port.

Client --- SYN ---> Server
Client <- SYN+ACK - Server
Client --- ACK ---> Server
Client <- Banner -- Server
Client -- RST+ACK > Server

If the daemon doesn't send a banner, nmap sends some probes to understand what the listening application is. It later tries to guess the application based on its behavior.

Network Discovery with port scanning

Sometimes, firewall can block pings. In this kind of scenario, typical usage of ping is insufficient. Even if host is not responding to ICMP requests, it oftentimes has some TCP and UDP port opens. nmap can be forced to scan host which is not responding to ping with -Pn. The scan for common ports like 22 (ssh), 80,443 (http/https server), 445 (samba service) or 53 (DNS) can reveal that the host is actually "alive" however not responding to ICMP.

Detecting firewall

Large networks very often are protected from intrusion by firewalls. It might be difficult to detect firewall, but based on partial or incomplete results of scan, one can identify, that it could be it. nmap scan with fingerprinting -sV should not have any problems to return full information. Sometimes, however, default information about VERSION could be not present or unrecognized (tcpwrapped means that TCP handshake was completed, but the remote host closed the connection without receiving any data). To try and see what happened, you might want to use nmap with --reason that will show an explanation of why a port is marked open or closed.

Masscan

Masscan was designed to deal with large networks and to scan thousands of IP addresses at once. It is similar to nmap but a lot faster.

Vulnerability Assessment

Vulnerability assessment is a scan of the vulnerabilities found on networks and applications. It is faseter and lighter on the infrastructure. As opposed to a penetration test, during a vulnerability assessment, you don't proceed to the exploitation phase. It means, that after discovering vulnerability, you won't be able to confirm it by testing and giving a proof of exploitation.

Scanners

Scanners perform their probes on:

  • daemons listening on TCP and UDP ports
  • configuration files of operating systems, software suites, network devices, etc.
  • windows registry entries.

The purpose is to find vulnerabilities and misconfigurations. The better and more up-to-date database of vulnerabilities, the better result of scan.

However, in a case of custom application, a vulnerability scanner may not be enough. In that case, manual test needs to be performed.

Resources

Oct 07, 2020

OSINT

Information gathering

First and the most crucial phase of an engagement is information gathering. It helps to broaden surface of attack, and prepare for successful method of breaking into company. Usually, people leave a lot of breadcrumbs around the Internet. Thanks to meticulous look up of the information, a lot of interesting details can be found.

Interesting databases

Given the vast array of social networks, it shouldn't be difficult to find interesting information. For example, user can create account on Twitter, point to Linkedin, Google and Facebook.

To find generic information, one might refer to twitter or google. For more work related information, usually better is Linkedin.

Crunchbase

Crunchbase is a platform for finding business information about private and public companies. It is a platform for finding business information about private and public companies. Additionally it contains useful information to match profiles.

Government websites

Some information can be found on official government websites. If a company is awarded contract.

Whois

whois is command line tool, as well as website. It allows to query the database of Internet addresses to get information about registrants of those addresses.

Company website

Oftentimes, interesting information can be found on company websites. We can find there information about products, customers, services, etc. Usually, there could be also find

Email pattern

Many companies have common email pattern. It usually means, that all employees, can be reached by some variation of email:

  • first_name.last_name@company.com
  • last_name.first_name@company.com
  • first_letter__nameLast_name@company.com

By accessing at least one of employees emails, you basically obtained all of them, as long as you know employee name.

To verify if chosen email is valid, one can send email and wait for response. Many email servers return message in the case of non-existing user.

Subdomain enumeration

It is common for a company, to have multiple different subdomains, like email.company.com, blog.company.com, careers.company.com. Sometimes, companies leave running vulnerable services. In that case, it could be much easier to exploit a bug on some outdated subdomain, than main domain. There are several ways of achieving that goal.

Google

One of the simplest ways is to search a Google! By issuing query to search engine like below, we can receive responses focused around particular company website.

site: company.com

dnsdumpster

DNSDumpster.com is a free domain research tool that can discover hosts related to a domain. Finding visible hosts from the attackers perspective is an important part of the security assessment process.

CLI Sublist3r

Sublist3r is a python tool designed to enumerate subdomains of websites using OSINT. It helps penetration testers and bug hunters collect and gather subdomains for the domain they are targeting. Sublist3r enumerates subdomains using many search engines such as Google, Yahoo, Bing, Baidu and Ask. Sublist3r also enumerates subdomains using Netcraft, Virustotal, ThreatCrowd, DNSdumpster and ReverseDNS.

Additional resources

Oct 01, 2020

Web Applications

Basics

During building web applications we need to focus on several fundamental aspects:

  1. HTTP Protocol Basics
  2. Cookies
  3. Sessions
  4. Same Origin Policy

HTTP Protocol Basics

HTTP (Hypertext Transfer Protocol) is the most used protocol on the Internet. Usually, the client (browser), connects to the server (Apache, nginx, ISS. During an HTTP communication, the client and the server exchange messages. HTTP works on top of TCP. That means, first a TCP connection is established, and then the client sends its request, and waits for the answer. The server processes the request and sends back its answer, providing a status code and appropriate data.

HTTP Request

To send HTTP request, usually we use browsers. However, it is not the only way of doing so. To build request, we can use netcat or telnet. Below is an example of HTTP request, with nc requesting website running on localhost on port 8000.

$ nc localhost 8000
GET / HTTP/1.0
Host: localhost

HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/3.7.3
Date: Fri, 02 Oct 2020 01:44:49 GMT
Content-type: text/html
Content-Length: 26612
Last-Modified: Fri, 02 Oct 2020 01:44:33 GMT

<!DOCTYPE html>
[...]

First line contains HTTP request method, path and protocol version. HTTP request method is information about the type of the request. Other requests are: PUT, POST, HEAD, etc. List of available metods can be found on Mozilla website Path tells the server which resource to fetch, while protocol version tells how to communicate with the client. Next, there is Host header field, which specifies the Internet hostname and port number of the resource being requested.

We can also create one-liner to retrieve appropriate information

$ echo -en 'HEAD / HTTP/1.1\r\nHost:localhost\r\n\r\n' | nc localhost 8000
HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/3.7.3
Date: Fri, 02 Oct 2020 02:01:34 GMT
Content-type: text/html
Content-Length: 27806
Last-Modified: Fri, 02 Oct 2020 01:55:46 GMT

Additionally, we can give extra header values like User-Agent (identifies the client and the system), Accept (specifies document type the client is expecting in the response) or Connection: keep-alive (future communications with the server will reuse the current connection).

HTTP Response

HTTP response, similar to request, has common format.

$ nc localhost 8000
GET / HTTP/1.0
Host: localhost

HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/3.7.3
Date: Fri, 02 Oct 2020 01:44:49 GMT
Content-type: text/html
Content-Length: 26612
Last-Modified: Fri, 02 Oct 2020 01:44:33 GMT

<!DOCTYPE html>
[...]

The first line is Status-Line which consists of protocol version (HTTP 1.0) followed by a numeric status code (200) and textual meaning (OK). There are many codes. Description, again can be found on Mozilla website. Next, there is additional information regarding server, date and time at which the message was originated, followed by page content.

HTTP Secure (HTTPS)

HTTP over SSL/TLS is a method to run clear-text HTTP with extra cryptographic protocol, to prevent from sniffing. By doing so, entire traffic is being encrypted. It means, that even if someone can intercept traffic, they won't be able to see what kind of information is being transmitted. The only non-encrypted pieces of information would be:

  • target IP address
  • target port
  • DNS or similar protocols (domain resolvers)

To analyze connection, we cannot use nc anymore, which doesn't support SSL. To work with SSL, we can use openssl

$ nc blog.smigiel.dev 443
HEAD / HTTP/1.1
$
$ openssl s_client -connect blog.smigiel.dev:443 -quiet
depth=2 O = Digital Signature Trust Co., CN = DST Root CA X3
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
verify return:1
depth=0 CN = blog.smigiel.dev
verify return:1
HEAD / HTTP/1.1
Host: blog.smigiel.dev

HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 30082
Server: GitHub.com
Content-Type: text/html; charset=utf-8
Last-Modified: Wed, 30 Sep 2020 03:07:04 GMT
ETag: "5f73f658-7582"
Access-Control-Allow-Origin: *
Expires: Fri, 02 Oct 2020 01:14:17 GMT
Cache-Control: max-age=600
X-Proxy-Cache: MISS
X-GitHub-Request-Id: 376E:4248:1F613:26B71:5F767C91
Accept-Ranges: bytes
Date: Fri, 02 Oct 2020 02:57:10 GMT
Via: 1.1 varnish
Age: 49
X-Served-By: cache-pao17443-PAO
X-Cache: HIT
X-Cache-Hits: 1
X-Timer: S1601607430.114992,VS0,VE1
Vary: Accept-Encoding
X-Fastly-Request-ID: b902c4a5a2f4410e8e6ac21e60fa2e659f1e8300

HTTP Cookies

HTTP is stateless protocol. It means, that website cannot keep the state of a visit across different HTTP requests. Every HTTP request is unrelated to others. To change this situation, cookies were introduced.

Cookies are textual information installed by a website into web browser. Server can set a cookie using Set-Cookie header field.

Cookie fields

Cookies are only sent to the valid domain and path when they are not expired and according to their flags. The domain field and the path field set the scope of the cookie. The browser sends the cookie only if the request is for the right domain. If the server does not specify domain attribute, the browser will automatically set the domain as the server domain and set the cookie host-only flag. It means that the cookie will be sent only to that precise hostname.

Cookie protocol

Cookies are usually set during a login. Browser sends POST request to the server, and the server responds with a Set-Cookie header field. For every subsequent request, the browser considers domain, path, expiration and flags. If all checks pass, the browser will insert a cookie: header in the request.

Sessions

Sessions mechanism works similar to cookies. The difference here relies on the way how the information is stored. Session is kept on server-side. Each user session is identified by a session id which the server assigns to the user. The client then presents its ID for each subsequent request, thus being recognized by the server. The server retrieves the state of the client and all its associated variables.

Same-Origin Policy (SOP)

Same-Origin Policy prevents JavaScript code from getting or setting properties on a resource comming from a different origin. To determine if JavaScript can access a resource hostname, port and protocol must match. SOP applies only to the actual code. It is still possible to include external resources by other HTML tags like img, script, iframe, object, etc.

Additional resources

Sep 23, 2020

Networking Pt 4

Firewall

Firewalls are specialized software modules running on a computer or a dedicated network device. They serve to filter packets coming in and out of a network. They can work on different layers of the OSI model.

Packet filtering

The most basic feature of a firewall is packet filtering. Administrator can create rules which will filter packets according to certain characteristics like:

  • source IP address
  • destination IP address
  • protocol
  • source port
  • destination port

Common actions, that can be applied:

  • allow: allow the packet to pass
  • drop: drops the packet without any diagnostic message to the packet source host
  • deny: similar to drop, but notify the source host

Inspecting the header of a packet does not give you any information on the content. Even if only port 80 or 443 is allowed, attacker can exploit them, to access deeper levels of network.

Application level firewalls

Application level firewall works by checking all 7 layers of the OSI model. They provide more comprehensive protection because they inspect the actual content of a packet, not just headers.

Intrusion Detection System (IDS)

IDS inspects the application payload trying to detect any potential attack. It checks for attack vectors like ping sweeps, port scans, SQL injections, buffer overflows etc. IDS, similar to antivirus, detects risky traffic by means of signatures. The vendor provides frequent signature updates as soon as new attack vectors are found. Without the right signatures an IDS cannot detect and report an intrusion. There is also risk of false positives when legic traffic is marked as malicious.

IDS != firewall IDS is another layer of protection, which can be used in conjunction with firewall.

Intrusion Prevention System (IPS)

It is similar to IDS, however it can drop malicious request when the threat has risk classification above predefined threshold.

Domain Name System (DNS)

The DNS converts human-readable names to IP addresses. DNS structure can be broken down into:

  • top level domain (TLD)
  • domain part
  • subdomain part (if applicable)
  • host part

Name resolution is performed by resolvers. Resolvers are servers that contact the TLD server and follow the hierarchy of the DNS name to resolve the name of a host:

  1. resolver contacts one of the root name servers, these server contain information about the TLD (this information is hardcoded by system administrator);
  2. next, it asks the TLD name server (from previous step), what's the name of the server which can give information about the domain the resolver is looking for;
  3. if there are one or more subdomains, previous step is repeated for every subdomain;
  4. resolver asks for the name resolution of the host part;
  5. the resolver sends the IP address back to the client.

Sep 21, 2020

Networking Pt 3: TCP and UDP

TCP & UDP

There are two common transport protocols used on the Internet: TCP (Transmission Control Protocol) and UDP (User Datagram Protocol). TCP is the most used transport protocol on the Internet. The vast majority of applications use it, and the IP protocol suite is often called TCP/IP. UDP is much simpler. It doesn't guarantee packet delivery and it is conectionless. Thanks to those features, it is much faster than TCP, it provides better throughput. It is often used by multimedia applications that can tolerate packet loss but are througput intensive. It is often used in VoIP and video streaming applications where you can tolerate a little glitch in the audio or video.

TCP UDP
Connection oriented protocol Connectionless
Rearranges data packets in the order specified No inherent order as all packets are independent of each other.
Slower speed than UDP UDP is faster, because error recovery is not attepmted. It is "best effort" protocol
Data is read as byte stream, no distinguishing information is sent to signal message boundaries Packets are sent individually and are checked for integrity only when they arrive. Packets have definite boundaries, meaning a read operation at the receiver socket will yield an entire message as it was originally sent.

Ports

Ports are used to identify a single network process on a machine. When some process is establishing connection to external internet resource (browser -> website), there is port open, associated with browser. Information about the port is later sent as part of header in TCP or UDP packet. In the case of aforementioned connection, source port would be the same, that was created for browser. Destination port then is associeated with website (usually 80 or 443). This is the way how process know where information needs to be addressed.

TCP handshake

In the case of UDP, which is connectionless, there is no handshake. However, for TCP, before any kind of transmission can happen, there needs to be established connection. It is done in so called three-way handshake. The header fields involved in the handshake are: sequence number, acknowledgement numbers, SYN and ACK flags

  1. The client sends a TCP packet to the server with the SYN flag enabled and a random sequence number.
  2. The server replies by sending a packet with both the SYN and ACK flags. Sequence number is again chosen as random, but ACK is seq number + 1.
  3. The client completes the synchronization by sending an ACK packet. The ACK is seq number + 1 and sequence number is ACK + 1 from the second step.
Host 1 | flags | Host 2
-------|-------|--------
-----> |SYN    |
    Seq: 328, ACK: 0

       |SYN/ACK| <------
    Seq: 412, ACK: 329

------>|ACK    |
    Seq: 330, ACK: 413

Additional resources

Next → Page 1 of 2