Build Time HackingMohammed Tanveer

21- March-2015

About me

Security Engineer at Citrix R&D India Pvt. Ltd.Gamer by the night and hacker by the day ;-)

Twitter: @threatpointer

Disclaimer: Thoughts and views expressed are of my own and does not resemble any of my employers!

Hacking during building Software!

Imagine we had the power to understand the code before its complied

How about embedding a backdoor ?

How about stealing legitimate certificates of a well known vendor and using them to sign malware?

Or even reading error logs which threw out a ton of useful internal details about the application itself!

Today- What to expect!

Explore frequently found Build Infrastructure design & implementation issues

Understand the threat landscape from the “build” point of view!

Gain better sight on assessing your existing Build infrastructure

Proposed ways on securing your build environment using existing tools and hardening guidelines.

About- Software Build

What is Build

From ToProcessof conversion

Source code Artifactsrun at the end-user

workstation

6

7

Why do we need to build ?

Configuration

BuildRESULT OF

defined set of components having specific versionscomponents = functional unitscomponents and their versions are chosen to meet specific

objectives or tasks

Main objectives

Functionality Performance

8

• Conclusively: To incorporate additional functionality (feature) To reach defined level of performance (as an example of non-

functional requirement)

Why do we need to build ?

Build tasks by priority

•Compilation• Deployment• DB integration• Unit testing• Code coverage• Static analysis• Source code metrics

• Dynamic analysis• Documentation generation

• …

9

Build types

• Classification by use:• Private [Developers build]• Integration [Centralized build]• Release [delivery to the end-user]

10

Build Tools

GNU Auto tools

•make•automake•autoconf•autoheader•libtool•gettext•gcc •pkg-config

Auto tools alternatives

•pymake•CMake•Cons•SCons•qmake•makepp•JAM•waf

Continuous Integration Tools•AnthillPro•Bamboo•Apache Continuum•BuildIT•BuildBot•Hudson•Jenkins•Team Foundation Server•TeamCity•Travis CI

Build Tools

GNU Auto tools

•make•automake•autoconf•autoheader•libtool•gettext•gcc •pkg-config

Auto tools alternatives

•pymake•CMake•Cons•SCons•qmake•makepp•JAM•waf

Continuous Integration Tools•AnthillPro•Bamboo•Apache Continuum•BuildIT•BuildBot•Hudson•Jenkins•Team Foundation Server•TeamCity•Travis CI

What is Continuous IntegrationContinuous Integration is a software development practice where members of a team integrate their work

frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each

integration is verified by an automated build (including test) to detect integration errors as quickly as possible.

--by Martin Fowler

Ref: http://martinfowler.com/articles/continuousIntegration.html

Continuous Integration Overview

Ref: http://www.javaworld.com/javaworld/jw-12-2008/images/CIOverview.jpg

Opportunities during Build Process

• External facing Assets GitHub/Subversion -SVN’s Build Engineer’s Tools – ex: Jenkins

• Internally facing Assets Source Code Repositories- Perforce, GIT CI Tools Monitoring Tools QA Testing Tools Code Signing Tools/keys

External facing –Opportunities

• Source code put up on Git/SVN is usually open to public, partially private or completely private

• Where is the opportunity ?1. Imagine a naïve developer contributes to an open source project with a

new feature, checks in code, project lead- reviews the feature/functionality and merges it the master repository

2. Invested efforts perform code audits to find bugs, often using them as ZeroDays

• This would imply to all projects that are completely open sourced or partially private ones.

What about completely Private ones ?

• We all know we don’t have access to them directly, right ?

The Indirect Way: (PenTester’s Way)• Check for all available build tools/assets that are being used by the

target vendor

Example: Jenkins

External facing –cont.

Build triggering and monitoring Tool- Jenkins

Jenkins an award winning CI tool and has the most active community in its category

We even have plenty of dorks

Jenkins-

Jenkins has had a very rich history of being PenTester friendly

Links for reference:https://www.pentestgeek.com/2014/06/13/hacking-jenkins-servers-with-no-password/

http://www.labofapenetrationtester.com/2014/06/hacking-jenkins-servers.htmlhttp://xn--thibaud-dya.fr/jenkins_credentials.html

http://dariusfreamon.wordpress.com/2014/01/07/jenkins-accountpasswordreset-account-name-enumeration/

Jenkins User Id’s

• Exposed Jenkins interface reveals information about the developers participating in the software projects. It is possible to extract user id’s of the registered accounts with associated names. This is a substantial point of gaining information about the users of a specific organization.

Software Builds Information and Source Codes

• Information about the changes in the software over a passage of time reveals information about the development flow of the components. Jenkins application does not ask for any explicit permission from the administrators to validate the user before accessing the ongoing software builds.

Server-side Information Disclosure in Environment Variables:

• Access to environment variables provides plethora of information. Example shows the significant information about the configuration for MySQL (JDBC) database server. The attacker can easily glean username and password of the database (SONAR_JDBC_USERNAME & SONAR_JDBC_PASSWORD) including the internal IP address with port number on which JDBC service is running

Server-side Information Disclosure in Environment Variables: -Count

Exposure of Application Secret Data

In this example, the curl command uses a secret key to connect to a specific domain for fetching JSON data through GET request.

The underlined example shows how a secret key is extracted by accessing the console-output.

CRON Job ExecutionThe example shown below presents the creation of database by a specific cron job in one of the vulnerable Jenkins configuration. It reveals how the table of OAUTH tokens is created and index is generated. The example shows the creation of database by a specific CRON job in one of the vulnerable Jenkins configuration. It reveals how the table of OAUTH tokens is created and index is generated.

Jenkins- Thick client issues

Jenkins comes with a command line utility which allows us to remotely manage builds on different nodes

The client jar file canbe modified to executecode on the remote Jenkins server

CLI Architecture & Quick Demo!Makes a http request on Port 8080

Looks for a CLI port in response headers

Returns the CLI port that has been configured to listen on

CLIENT SERVER

An http request is sent on the specific port

ELSE

Full Duplex http stream

CLIENT SERVER

Jenkins – Groovy Script

• Jenkins features a nice Groovy script console which allows to run arbitrary scripts on the Jenkins server or on slave nodes. This feature can be accessed from the "manage Jenkins" link, typically at your http://server/jenkins/script

MetaSploit Love

• Use the Auxiliary mode to understand the Jenkins configurations• Using Jenkins Script-Console Java Execution

• exploit/multi/http/jenkins_script_console

Jenkins- Plugins

• SonarQube• SonarQube Plugin for Jenkins contains a flaw that is triggered when a direct

request is sent for /jenkins/configure, which will cause the plugin to display password information via the 'sonar.sonarPassword ' parameter. This may allow a remote authenticated attacker to gain access to password information

• Active Directory• Jenkins contains an unspecified flaw in the Active Directory plugin that may

allow an attacker to gain unauthorized administrator access• Open ID

• Jenkins Open ID Plugin contains an unspecified flaw in SSO mode that may allow an attacker to gain unauthorized administrative access

Plugins

Hudson Tray Application

• This plugin provides the Hudson User a tool that runs on their desktop, that monitors the status of the Hudson Server (multiple servers supported), and provides a simple tray icon on the Notification Panel/System Tray, showing them the worst case color of the Hudson Jobs they are monitoring

The same plugin allows you to upload arbitrary python scripts to the server.

Lesson learnt!!

We make use of the so call features of tools that are either misconfigured or mismanaged leading us to compromise the complete Build environment.

Problems known- Solutions ?

Dual Jenkins -DMZ based deployment for public facing interfaces

Problems known- Solutions ?

• Take special care of all tools interacting with the build environment.• Disable as many plugins as possible- reducing the overall attack

surface• If a public facing Jenkins is strictly needed- Use the DMZ method

explained in the previous slide.• Username used should not be same as corporate id’s, these are

anyways going to be public • Old school- Patch them regularly and check logs regularly

Speaking of Internally facing assets

• Source Code Repositories- Perforce, GIT• CI Tools• Monitoring Tools• QA Testing Tools • Code Signing Tools/Private keys• Build Engineers Systems

Internally facing - Solutions

Internally facing - Solutions –cont.

• Make sure all components interacting with the build environment is patched regularly and checked for any misconfigurations

• Its best to have a separate domain to operate your Build environment & create groups specifically for Build engineers only

• Should you have multiple teams/products, segregate them within the protected environment using dedicated VLAN’s

• Define Firewall rules for only what is needed and block the rest.

Credits

• Ahmed Gomaa• Nikhil SamratAshok Mittal• James Luther

Thank You!