Last night, we got hit by a ~3 Gbps DDoS attack. It's been a while since this has happened to us, so I'd like to make sure I'm still up on the state of the art. Is there anything more to be done than the following? 1. Identify the victim. 2. Null route the victim. 3. Propagate the null route to your upstreams (via BGP, if supported, otherwise a phone call to their NOC). 4. Move the victim to a new IP. To avoid participating in at least some classes of DDoS attacks, we: * long ago implemented uRPF (and/or similar ACLs) to block spoofed outbound packets, as recommended by BCP 38 (RFC 2827). * ensured our NTP servers (and any NTP servers of our customers) are not responding to monlist queries. The openntpproject.org website is useful here. They list vulnerable NTP servers by IP range, or you can get all NTP servers by AS (replace YOUR_AS with your AS, and optionally, add &csv=1) and then query with ntpdc -n -c monlist IP: http://openntpproject.org/searchby-asn.cgi?search_asn=YOUR_AS * just this week started addressing customers with open DNS resolvers, which can also be used in amplification attacks: http://openresolverproject.org/searchby-asn.cgi?search_asn=YOUR_AS Is there anything else we should be doing? -- Richard ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
hi all, there’s a recent report from ICANN’s SSAC (Security and Stability Advisory Committee) on DDoS attacks using DNS infrastructure that may be useful https://www.icann.org/en/groups/ssac/documents/sac-065-en.pdf here’s the Executive Summary to give you a sense of what’s in it. Executive Summary This document is intended for the Internet technical community, particularly authoritative and recursive Domain Name System (DNS) operators, network operators, the Internet Corporation for Assigned Names and Numbers (ICANN), and policy makers. It explores several unresolved critical design and deployment issues that have enabled increasingly large and severe Distributed Denial of Service (DDoS) attacks using the DNS. While DDoS attacks can exploit multiple characteristics of network infrastructure and operations, the prevalence and criticality of the DNS means that securing it is both challenging and urgent. These unresolved DNS issues and related DDoS attacks pose a real and present danger to the security and stability of the Internet. The first recommendation below is made to ICANN, while others are made to operators of Internet infrastructure and manufacturers. While in many instances they reflect actions not under ICANN’s control and actors not necessarily within ICANN’s usual community, they are meant to address the overall responsibilities of the multi-stakeholder community and encourage ICANN to take action where it is relevant to do so. In particular, this means ICANN should be looking for ways to increase the effectiveness of steps already being taken against DNS abuse and promoting the participation of others as well as pursuing the measures suggested here. The Security and Stability Advisory Committee (SSAC) strongly recommends that all types of DNS operators and network operators take immediate steps to mitigate the design and deployment issues that make large scale DDoS attacks possible. Specifically, the SSAC strongly recommends that: • ICANN should help facilitate an Internet-wide community effort to reduce the number of open resolvers and networks that allow network spoofing. This effort should involve measurement efforts and outreach. • All network operators should take immediate steps to prevent network address spoofing. • Recursive DNS server operators should take immediate steps to secure open recursive DNS servers. • Authoritative DNS server operators should support efforts to investigate authoritative response rate limiting. • DNS server operators should put in place operational processes to ensure that their DNS software is regularly updated and communicate with their software vendors to keep abreast of the latest developments. • Manufacturers and/or configurators of customer premise networking equipment, including home networking equipment, should take immediate steps to secure these devices and ensure that they are field upgradable when new software is available to fix security vulnerabilities, and aggressively replace the installed base of non-upgradeable devices with upgradeable devices. On Mar 13, 2014, at 2:04 PM, Richard Laager <rlaager@WIKTEL.COM> wrote:
Last night, we got hit by a ~3 Gbps DDoS attack. It's been a while since this has happened to us, so I'd like to make sure I'm still up on the state of the art.
Is there anything more to be done than the following? 1. Identify the victim. 2. Null route the victim. 3. Propagate the null route to your upstreams (via BGP, if supported, otherwise a phone call to their NOC). 4. Move the victim to a new IP.
To avoid participating in at least some classes of DDoS attacks, we: * long ago implemented uRPF (and/or similar ACLs) to block spoofed outbound packets, as recommended by BCP 38 (RFC 2827). * ensured our NTP servers (and any NTP servers of our customers) are not responding to monlist queries. The openntpproject.org website is useful here. They list vulnerable NTP servers by IP range, or you can get all NTP servers by AS (replace YOUR_AS with your AS, and optionally, add &csv=1) and then query with ntpdc -n -c monlist IP: http://openntpproject.org/searchby-asn.cgi?search_asn=YOUR_AS * just this week started addressing customers with open DNS resolvers, which can also be used in amplification attacks: http://openresolverproject.org/searchby-asn.cgi?search_asn=YOUR_AS
Is there anything else we should be doing?
-- Richard
########################################################################
To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
PHONE: 651-647-6109, FAX: 866-280-2356, WEB: www.haven2.com, HANDLE: OConnorStP (ID for Twitter, Facebook, LinkedIn, etc.) ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
I've documented over 60 attacks to customers in our network in the last 22 months. Some are small, just 200 Mbps, the largest has been 2.6 Gbps. In 98% of all cases the attack is just six to eight minutes long, so by the time I respond to any alarms it's already over. So there's one more thing you can do. 5. Increase your uplink so that the small attacks end up affecting only the subset of access technology that's part of the targeted customer. Frank -----Original Message----- From: MICE Discuss [mailto:MICE-DISCUSS@LISTS.IPHOUSE.NET] On Behalf Of Richard Laager Sent: Thursday, March 13, 2014 2:04 PM To: MICE-DISCUSS@LISTS.IPHOUSE.NET Subject: [MICE-DISCUSS] DDoS Attacks Last night, we got hit by a ~3 Gbps DDoS attack. It's been a while since this has happened to us, so I'd like to make sure I'm still up on the state of the art. Is there anything more to be done than the following? 1. Identify the victim. 2. Null route the victim. 3. Propagate the null route to your upstreams (via BGP, if supported, otherwise a phone call to their NOC). 4. Move the victim to a new IP. To avoid participating in at least some classes of DDoS attacks, we: * long ago implemented uRPF (and/or similar ACLs) to block spoofed outbound packets, as recommended by BCP 38 (RFC 2827). * ensured our NTP servers (and any NTP servers of our customers) are not responding to monlist queries. The openntpproject.org website is useful here. They list vulnerable NTP servers by IP range, or you can get all NTP servers by AS (replace YOUR_AS with your AS, and optionally, add &csv=1) and then query with ntpdc -n -c monlist IP: http://openntpproject.org/searchby-asn.cgi?search_asn=YOUR_AS * just this week started addressing customers with open DNS resolvers, which can also be used in amplification attacks: http://openresolverproject.org/searchby-asn.cgi?search_asn=YOUR_AS Is there anything else we should be doing? -- Richard ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1 ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
On 3/13/14, 14:04 , Richard Laager wrote:
Last night, we got hit by a ~3 Gbps DDoS attack. It's been a while since this has happened to us, so I'd like to make sure I'm still up on the state of the art.
Is there anything more to be done than the following? 1. Identify the victim. 2. Null route the victim. 3. Propagate the null route to your upstreams (via BGP, if supported, otherwise a phone call to their NOC). 4. Move the victim to a new IP.
3a. Notify your (MICE) peer if they are participating in the attack. We for one want to respond and deal with issues to the extent possible. FYI, specifically for us abuse@umn.edu, sometimes we are not the quickest, but we really don't just ignore issues reported to us.
To avoid participating in at least some classes of DDoS attacks, we: * long ago implemented uRPF (and/or similar ACLs) to block spoofed outbound packets, as recommended by BCP 38 (RFC 2827).
Yes, please implement BGP38 to the extent possible, we do. Yes, not everyone realistically can, but if everyone that can did, the world would have a better Internet.
* ensured our NTP servers (and any NTP servers of our customers) are not responding to monlist queries. The openntpproject.org website is useful here. They list vulnerable NTP servers by IP range, or you can get all NTP servers by AS (replace YOUR_AS with your AS, and optionally, add &csv=1) and then query with ntpdc -n -c monlist IP:
We addressed this in January for our authoritative public NTP servers. But, there were a lot of other NTP servers on campus, this took us a while to mitigate. We think we have this squelched now, but let us know if you see moles that need whacking, especially if the mole is bothering you.
http://openntpproject.org/searchby-asn.cgi?search_asn=YOUR_AS * just this week started addressing customers with open DNS resolvers, which can also be used in amplification attacks: http://openresolverproject.org/searchby-asn.cgi?search_asn=YOUR_AS
A continual battle for years, if not a decade. Again, let us know if you see moles that need whacking, especially if the mole is bothering you. :(
Is there anything else we should be doing?
If you don't already, block other UDP ports that probably shouldn't talk to the Internet because they are also targets of UDP amplification attacks too, SNMP, Chargen, SSDP, QOTD, etc... https://www.us-cert.gov/ncas/alerts/TA14-017A -- ================================================ David Farmer Email: farmer@umn.edu Office of Information Technology University of Minnesota 2218 University Ave SE Phone: 1-612-626-0815 Minneapolis, MN 55414-3029 Cell: 1-612-812-9952 ================================================ ######################################################################## To unsubscribe from the MICE-DISCUSS list, click the following link: http://lists.iphouse.net/cgi-bin/wa?SUBED1=MICE-DISCUSS&A=1
participants (4)
-
David Farmer
-
Frank Bulk
-
Mike O'Connor
-
Richard Laager