Payment Card Security Breach

I managed the PCI Data Security Standards compliance program for a regional supermarket chain last year, so I've been following the recent breach at a similar company on the east coast (Hannaford Brothers) with considerable interest. My notes are provided below.

About the Breach:

• Hannaford & Sweetbay (a related banner) have 271 stores; all were affected, including 23 independent stores (294 total).
• 4.2M cards were breached; 1,800 cases of fraud identified so far; 2nd largest known merchant breach to date.
• Breach started on 12/7/07; VISA notified Hannaford on 2/27/08; breach was contained on 3/10/08; press release issued on 3/17/08.
• Hannaford conducted (and passed) their 2nd PCI audit while the breach was in progress (in February).
• Sophisticated network attack "during transmission of card authorization".
• Malicious software was installed on the point-of-sale (POS) servers in all of the stores. The software was custom-tailored to work on an IBM POS server to read track 2 data from cards during the authorization process. It batched up a number of records and transmitted them periodically to an unspecified server at an overseas Internet service provider.
  
• Hannaford web site is almost entirely in maintenance mode (possible attack vector?)
• Most likely source of breach: an "insider" attack (employee, consultant, contractor, vendor support, etc.). Organized crime may have gotten to someone who has routine access inside their PCI firewall.

Point-of-Sale Environment:

• Hannaford stores do not use wireless in their point-of-sale infrastructure; also, they upgraded their wireless last year.
• IBM WebSphere MQ is used for financial transactions; this software is sometimes misconfigured to allow read access to transaction queues; it also implies that Hannaford uses a Linux-based point-of-sale system from IBM.
• Hannaford uses centralized EFT switch software from ACI.
• Hannaford's acquirer is First Data; transactions are sent there for processing.

Hannaford Response:

• Ron Hodge, CEO, offered a public apology in the press release.
• Carol Eleazer, VP Marketing, provided most information about the breach in the first few days after the press release.
• Emily Dickinson, General Counsel, released more specific details on the nature of the attack.
• Hannaford set up a special 24/7 call center staffed with 25 people in 12-hr shifts to take customer calls.
• Internal experts and outside consultants worked around the clock for 12 days to identify and contain the breach.
• They don't know how the malicious software got onto the POS servers, but they managed to stop the leak on 3/10/08 by replacing all affected hardware.

Related Impacts:

• Depending on the outcome of forensic analysis (were they in fact compliant during the breach?), they may see $1M fine from VISA and MasterCard. Merchants have safe harbor from such fines if they're PCI compliant, but they still have other liabilities if they're breached.
• There are a couple of controls in the PCI Data Security Standards that should have caught this, implying that Hannaford may have had some holes in their PCI compliance: (1) egress filters on the PCI firewall should have blocked the transmissions to a server on the Internet; and (2) a file integrity monitoring tool (such as Tripwire) should have detected the installation of the malicious software. Perhaps no one was watching the monitoring system.
• Class action lawsuits have been filed, including one by the same firm that sued TJX.
• State Attorneys General have complained about not being notified, but no personal information was disclosed, which would trigger their reporting laws. Only track 2 data (which does not include customer name) was breached.
• Security equipment vendor Rapid7 pulled their Hannaford customer references and suffered a major PR problem. As an aside, this is interesting evidence that marketing departments cannot control the message; consider the Cluetrain theses.
  
• Hannaford's auditor has not been disclosed, but they've got some explaining to do. Their contract probably mandates that they may not publicly disclose who their auditor is, in anticipation of events just like this.

Impact on PCI:

• Organized crime has reportedly been shifting their focus from data at rest to data in transit as companies stop storing full track data and encrypt data on disk.
• This was the first significant theft of data in transit (sniffed off the network versus read off the hard drive). Note that network-level encryption would probably not have prevented this breach. Message-level encryption from the PIN pad or register might have prevented it.
  
• This was the first significant theft from a PCI-compliant merchant, calling the adequacy of the standards into question.
• Impact on PCI standards: they will likely require encryption for data in transit even on private networks. This will impact nearly every merchant and every payment-processing service provider.

Related News:

• Monday, March 17, 2008
• Tuesday, March 18, 2008
• Wednesday, March 19, 2008
• Thursday, March 20, 2008
• Friday, March 27, 2008

Fixing the Problem

I think the entire payment processing system needs to be reevaluated. We need to redefine the problem. The old problem statement might be expressed as "all the points in the transaction chain of custody need to be secured". A better problem statement might be "it's too easy to charge something to someone's financial account; all you need is a small amount of information". Security professionals derisively call this "security by obscurity".

Instituting PIN numbers on credit cards like we have on debit cards might be a reasonable first step. It would be relatively simple compared to other alternatives, and would take care of some of the fraud. Note that VISA's PayWave is a step in the wrong direction, particularly when a PIN or some other authentication factor is not required.

A better solution would be to build a new payment processing network with a three-party architecture (Kerberos-like) that provides (1) positive multi-factor authentication of both merchant and customer identity, and (2) non-repudiation features (such as those provided by the AS2 protocol) so that neither party could say that a specific transaction did not take place. The fees could be much lower, because fraud and chargebacks could be virtually eliminated. The substantially lower transaction processing fees would provide incentive for merchants to upgrade their infrastructure to accommodate the new payment processing network. The new system would be very expensive to implement, but considering the cost of fraud and cost of upgrading the security of every point in the existing transaction processing network in the desperate attempt to protect the obscurity of a scrap of information, it would be worth it.

Update: on Friday, March 27, the Boston Globe released additional information on the nature of the security breach (malicious software installed on point-of-sale servers in each of the stores). I updated my notes in this post to include the new information.