Palo Alto : DNS Sinkhole

The Problem:

We have a infected user and that user is trying to reach out to a command and control server, the infected user does a DNS lookup and since this domain is not hosted locally the internal DNS will pass the request through the Firewall to the external DNS server , the logs wont give all the information we need.

We are going to intercept the DNS traffic between the Internal and External DNS server and respond with a DNS server of our own. Palo Alto send these DNS requests from the infected machines to 72.5.65.111 , which is a Palo Alto assigned address, that will force the traffic to the Firewall to be blocked and logged appropriately.

You do need a Threat Prevention License.

The antivirus release notes will list all the domains that Palo Alto deem to be suspicious.

This is only needed for traffic going to the internet.

How to Configure DNS Sinkhole

Make sure the latest Anti-Virus updates are installed. Device > Dynamic Updates > Click “Check Now”

Configure DNS Sinkhole in the Security Profile Anti-Spyware . Objects > Anti-Spyware under Security Profiles.

Create a New Anti-Spyware Profile or Use an existing one.

Change Action to “sinkhole”

Set Sinkhole IPv4 to the address mentioned above 72.5.65.111
Set Sinkhole IPv6 to the address mentioned above ::1

You then have to apply this security profile to your outbound internet Security Policy/Rule. Select the Rule > Actions > Choose Anti-Spyware Profile

If you want to log who is hitting the sinkhole address you will need to create a deny rule.

 

Commit the Config

Test Connectivity with Telnet

Telnet is a Network protocol that allows users to connect to and administer a devices Command Line Interface (CLI). However all of the information exchanged on a Telnet session is unencrypted, this means is someone is sniffing the traffic from your host to the device it can be read clearly. So now all know telnet is a super un-secure way to access the CLI of a device, and you should always use SSH where you can, but Telnet can be used in another way. Telnet can only be used to verify network connectivity to remote devices that are TCP based, because TCP is a connection-oriented protocol.

I get requests all the time to open up specific ports on the firewall. You can find a list of the well know port numbers here.

Before I start poking holes in the firewall or other device I check to see if it is already open.

Telnet will by default only check and listen on TCP port 23

If a user asks me is https (port 443) is open to and on a specific server you can easily test with Telnet.

You simply add the port number at the end of the telnet command:

 telnet[host/address[port]]

 

As you can see 443 is open. To exit from the Telnet session: 

SHIFT ]

 

This telnet test to port 23 is not open as it does not say Connected to…


If a remote host does not respond to telnet it can mean a number of things.

  1. The Firewall or Firewalls or other networking devices on the path to the remote host is Denying or Dropping the packets. You will be able to confirm that in the logs on the firewall.
  2. The server or remote host on the other side is not up and active
  3. There is no connectivity to the remote host for some other reason

Further troubleshooting is required if you encounter issues.

The Packet Wizard : VPN Split-Tunneling

Split-tunneling is a networking approach that lets a remote user using Remote Access Virtual Private Network (RAVPN) to have specific traffic sent to the internet instead of being sent over the encrypted VPN tunnel.

E.g. – A remote user is using a home network, hotel network or coffee shop to Remote Access VPN  (RAVPN) to connect to their works corporate network . The user or VPN subnet with split tunneling enabled can allow the user to send specific traffic such as; access to company file stores, company database servers, company mail servers and other servers on the corporate resources through the RAVPN connection. When the user connects to Internet resources such as Web sites, Personal Webmail, Voice or Video calls, etc.), the connection request can be sent directly out the local gateway provided by the home network, hotel network or coffee shop, thus preventing the traffic from being sent to the corporate network to be redirected to the internet, instead just going directly to the internet.

There are some Advantages of Split-Tunneling can be preventing bottlenecks especially if the user uses Voice/Video calls, where the calls can be severely depredated due to having to pass through the VPN tunnel first.

There are also some disadvantages of Split-Tunneling in that the user now by-passes Corporate security controls  that may be in place by the Security team for access to specific sites etc.

Cisco : Enable SSH on Cisco Switch, Router and ASA

When you configure a Cisco device, you need to use a console cable and connect directly to the system to access it. Follow the SSH setup below, will enable SSH access to your Cisco devices, since SSH is not enabled by default. Once you enable SSH, you can then access it remotely using SecureCRT or any other SSH client.

Set hostname and domain-name

The hostname has to have a hostname and domain-name.

switch# config t
switch(config)# hostname tpw-switch
tpw-switch(config)# ip domain-name thepacketwizard.com

Setup Management IP

In the following example, the management ip address will be set to 10.100.101.2 in the 101 VLAN. The default gateway points to the firewall, which is 10.100.101.1

tpw-switch# ip default-gateway 10.100.101.1
tpw-switch# interface vlan 101
tpw-switch(config-if)# ip address 10.100.101.2 255.255.255.0

Generate the RSA Keys

The switch or router should have RSA keys that it will use during the SSH process. So, generate these using crypto command as shown below.

tpw-switch(config)# crypto key generate rsa
  The name for the keys will be: tpw-switch.thepacketwizard.com
  Choose the size of the key modulus in the range of 360 to 2048 for your
    General Purpose Keys. Choosing a key modulus greater than 512 may take
    a few minutes.

How many bits in the modulus [512]: 1024
  % Generating 1024 bit RSA keys, keys will be non-exportable...[OK]

Setup the Line VTY configurations

Setup the following line vty configuration, where input transport is set to SSH only. Set the login to local, and password to 7, and make sure Telnet is not enabled:

tpw-switch# line vty 0 4
 tpw-switch(config-line)# transport input ssh
 tpw-switch(config-line)# login local
 tpw-switch(config-line)# password 7
 tpw-switch(config-line)# exit

If you have not set the console line yet, use the following:

tpw-switch# line console 0
tpw-switch(config-line)# logging synchronous
tpw-switch(config-line)# login local

Create the username password

If you don’t have an username created already, here is how:

tpw-switch# config t
Enter configuration commands, one per line.  End with CNTL/Z.
tpw-switch(config)# username thepacketwizard password tpwpassword123
tpw-switch# enable secret tpwenablepassword

Make sure the password-encryption service is turned-on, which will encrypt the password, and when you do “show run”, you’ll see only the encrypted password and not clear-text password.

tpw-switch# service password-encryption

Verify SSH access

From the switch, if you do ‘show ip ssh’, it will confirm that the SSH is enabled on this Cisco device.

tpw-switch# show ip ssh
 SSH Enabled - version 1.99
 Authentication timeout: 120 secs; Authentication retries: 3

After the above configurations, login from a remote machine to verify that you can ssh to this cisco switch.

In the example, 10.100.101.2 is the management ip-address of the switch.

TPW-Remote-Computer# ssh 10.100.101.2
 login as: thepacketwizard
 Using keyboard-interactive authentication.
 Password:

tpw-switch>en
 Password:
 tpw-switch#

You are now setup and logged in on SSH!

To read more on SSH visit: https://en.wikipedia.org/wiki/Secure_Shell

Palo Alto : Upgrade High Availability (HA) Pair

Over the last 3 weeks since the Christmas and New Year Holidays,  I have been upgrading all of our firewalls globally, many of them are an High Availability Pair. This means they are redundant and being redundant allows me to upgrade them individually while the site stays full up and functional.

The  instructions for upgrading an HA pair are recommended because:

      • It verifies HA functionality before starting the upgrade.
      • It ensures the upgrade is successfully applied to the first device before starting the upgrade on the second.
      • At any point in the procedure, if any issue arises, the upgrade can be seamlessly reverted without any expected downtime (unless you are having any dynamic routing protocols line OSPF/BGP).
      • When finished, the final active/passive device state will be the same as it was before the upgrade with the fewest number of fail overs possible (2).

Before you Begin :

Take backup of the configuration as well as Tech Support from both HA Peers. Give proper names to each file, here is how:

Device > Setup > Operations > Save Named Configuration Snapshot

Device > Setup > Operations > Export Named configuration Snapshot

Device > Setup > Operations > Export Device State (If device managed from panorama

Device > Support > Generate Tech Support File, and then download it. (Might be required if any issues)

(Optional but recommended) Disable preemption on High Availability settings to avoid the possibility of unwanted failovers. Disabling preempt configuration change must be committed on both peers. Likewise, once completed, re-enabling must be committed on both peers.

To disable preempt, go to

Device > High Availability > Election Settings and uncheck Preemptive.
Then, perform a commit.

 

 

 

If upgrade is between major versions (4.1 -> 5.0 OR 5.0-> 6.0), it is advisable to disable TCP-Reject-Non-SYN, so that sessions can failover even when they are not in sync. : Do this on both Firewalls from the CLI:

 # set deviceconfig setting session tcp-reject-non-syn no
 # commit

 

(Optional but recommended) Arrange for Out-of-Band access (Console access) to the firewall if possible. This is again to help recover from any unexpected situation where we are unable to login to the firewall. If you have a Terminal Server awesome, if not a simple Cell Phone tethered to a Laptop with RDP is also fine.

 

The Upgrade Process

Suspend Backup Device 

From the CLI

 > request high-availability state suspend

From the GUI

Go to Device > High Availability > Operational Commands  > Suspend local device

Install the new PAN-OS on the suspended device

Device > Software > Install

Reboot the device to complete the install.

When the upgraded device is rebooted, check the dashboard to check the version, wait for all the interfaces to come backup green.

If the device is still in suspended state make it functional again

From the CLI

> request high-availability state functional

From the GUI

Go to Device > High Availability > Operational Commands  > Make Local Device Functional

 

Repeat steps on other firewall.

 

Suspend Primary Device

From CLI

> request high-availability state suspend

From the GUI

Go to Device > High Availability > Operational Commands  > Suspend local device.

 

*The Backup Firewall will become Active – it does take 30-45 seconds so don’t panic

 

Install the new PAN-OS on the suspended device:

Device > Software > Install

Reboot the device to complete the install.

When the upgraded device is rebooted, check the dashboard to check the version, wait for all the interfaces to come backup green.

If the device is still in suspended state make it functional again

From the CLI

> request high-availability state functional

From the GUI

Go to Device > High Availability > Operational Commands  > Make Local Device Functional

To Get Primary Back to Primary by suspending the backup (current active) firewall (The Original Backup Firewall)

From the GUI,

Go to Device > High Availability > Operational Commands  > Suspend local device.

Once the Primary became active again, enable the suspended backup firewall

Enable TCP-Reject-Non-SYN, so that sessions can failover even when they are not in sync. : (Do this on both Firewalls)

# set deviceconfig setting session tcp-reject-non-syn yes
# commit

 

Re-Enable preempt configuration change must be committed on both peers. To re-enable preempt, go to Device > High Availability > Election Settings and uncheck Preemptive.  Then, perform a commit.

 

How to Downgrade

If an issue occurs on the new version and a downgrade is necessary:

To revert to the previous PAN-OS screen, run the following CLI command:

# debug swm revert

This causes the firewall to boot from the partition in use prior to the upgrade. Nothing will be uninstalled and no configuration change will be made.

 

However please be aware while running this command –

After rebooting from a SWM revert, the configuration active at the time before upgrade will be loaded with the activation of the previous partion. Any configuration changes made after upgrade will not be accounted for and will need to be manually recovered by loading the latest configuration version and committing the changes.

Palo Alto : Reconnaissance Protection Whitelist

Recently I have been implementing a software called Insight VM by Rapid 7 which runs reconnaissance on our network looking for vulnerabilities. Whilst this software is scanning, I was finding the Firewall would block it (like its supposed to) and then complain like crazy that it and its Network was being targeted. 27,000 email over night I decided to research how to solve this issue. Luckily Palo Alto have thought about this.

Here is how to implement Reconnaissance Protection Whitelist:

Select Network>Network Profiles>Zone Protection>Reconnaissance Protection to add a source address exclusion whitelist to your zone protection Profile.

Add an address to your source address exclusion whitelist. You add up to 20 IP addresses or netmask address objects.

Solarwinds : Download “Set” Command backup for Palo Alto

Download “Set” Command  backup for Palo Alto

Settings > All Settings > NCM Settings > Advanced > Device Templates > Search ‘Palo Alto’ > Select ‘PaloAlto5050’ > Click Copy > Change Template Name ‘PaloAlto5050 – Set’ > Remove the XML information and then Copy and Paste the XML below > Click Save

<Configuration-Management Device="Palo Alto" SystemOID=" 1.3.6.1.4.1.25461.2.3">
 <Commands>
 <Command Name="RESET" Value="set cli pager off${CRLF}set cli config-output-format set${CRLF}configure" RegEx="#" />
 <Command Name="EnterConfigMode" Value="" />
 <Command Name="ExitConfigMode" Value="exit" />
 <Command Name="Startup" Value="saved running-configuration" />
 <Command Name="Running" Value="" />
 <Command Name="DownloadConfig" Value="show ${ConfigType}" />
 <Command Name="UploadConfig" Value="" />
 <Command Name="DownloadConfigIndirect" Value="" />
 <Command Name="SaveConfig" Value="commit" />
 <Command Name="Version" Value="show" />
 </Commands>
 </Configuration-Management>

 

Change the Template being used on the Palo Alto Nodes

Settings > Manage Nodes > Palo Alto > Select All > Edit Properties > Tick Communication > Select Device Template ‘Palo Alto5050 – Set’ > Submit.

Data Centre : Post DC Move Unracking

We moved our company internal Data Centre to a COLO Facility 2 weeks ago, here is what is left. Before and After Pictures, as well as a photo of the “Boneyard”. A pretty good haul for E-Wasting:

2x Cisco 6909’s

3x Cisco 6513’s

8x Cisco ASA’s

2x Brocade Loadbalancers

4x Cisco 2900 Routers

2x Cisco Nexus 5k

1x Cisco Wireless LAN Controller

 

Palo Alto : Enable IPv6 and Create Default Route

To Enable IPv6 on the Firewall

Web GUI

IPv6 firewalling is enabled under Device > Setup > Session:

*** You may have to restart your Firewall for IPv6 to be enabled.

On the CLI

> configure

# set deviceconfig setting session ipv6-firewalling [yes|no]

# commit

Here is the interface configuration I used:

Don’t forget to add a rule in your security policy that allows Your new IPv6 interface to talk to your Gateway.

Add Default Route

You will also have to add a default route under Network > Virtual Routers > Default > Static Routes > IPv6

The default route for IPv6 is ::/0 the next hop is the default gateway address