Payment Card Security Breach

I managed the PCI Data Security Standards compliance program for a regional supermarket chain last year, so I've been following the recent breach at a similar company on the east coast (Hannaford Brothers) with considerable interest. My notes are provided below.

About the Breach:

• Hannaford & Sweetbay (a related banner) have 271 stores; all were affected, including 23 independent stores (294 total).
• 4.2M cards were breached; 1,800 cases of fraud identified so far; 2nd largest known merchant breach to date.
• Breach started on 12/7/07; VISA notified Hannaford on 2/27/08; breach was contained on 3/10/08; press release issued on 3/17/08.
• Hannaford conducted (and passed) their 2nd PCI audit while the breach was in progress (in February).
• Sophisticated network attack "during transmission of card authorization".
• Malicious software was installed on the point-of-sale (POS) servers in all of the stores. The software was custom-tailored to work on an IBM POS server to read track 2 data from cards during the authorization process. It batched up a number of records and transmitted them periodically to an unspecified server at an overseas Internet service provider.
  
• Hannaford web site is almost entirely in maintenance mode (possible attack vector?)
• Most likely source of breach: an "insider" attack (employee, consultant, contractor, vendor support, etc.). Organized crime may have gotten to someone who has routine access inside their PCI firewall.

Point-of-Sale Environment:

• Hannaford stores do not use wireless in their point-of-sale infrastructure; also, they upgraded their wireless last year.
• IBM WebSphere MQ is used for financial transactions; this software is sometimes misconfigured to allow read access to transaction queues; it also implies that Hannaford uses a Linux-based point-of-sale system from IBM.
• Hannaford uses centralized EFT switch software from ACI.
• Hannaford's acquirer is First Data; transactions are sent there for processing.

Hannaford Response:

• Ron Hodge, CEO, offered a public apology in the press release.
• Carol Eleazer, VP Marketing, provided most information about the breach in the first few days after the press release.
• Emily Dickinson, General Counsel, released more specific details on the nature of the attack.
• Hannaford set up a special 24/7 call center staffed with 25 people in 12-hr shifts to take customer calls.
• Internal experts and outside consultants worked around the clock for 12 days to identify and contain the breach.
• They don't know how the malicious software got onto the POS servers, but they managed to stop the leak on 3/10/08 by replacing all affected hardware.

Related Impacts:

• Depending on the outcome of forensic analysis (were they in fact compliant during the breach?), they may see $1M fine from VISA and MasterCard. Merchants have safe harbor from such fines if they're PCI compliant, but they still have other liabilities if they're breached.
• There are a couple of controls in the PCI Data Security Standards that should have caught this, implying that Hannaford may have had some holes in their PCI compliance: (1) egress filters on the PCI firewall should have blocked the transmissions to a server on the Internet; and (2) a file integrity monitoring tool (such as Tripwire) should have detected the installation of the malicious software. Perhaps no one was watching the monitoring system.
• Class action lawsuits have been filed, including one by the same firm that sued TJX.
• State Attorneys General have complained about not being notified, but no personal information was disclosed, which would trigger their reporting laws. Only track 2 data (which does not include customer name) was breached.
• Security equipment vendor Rapid7 pulled their Hannaford customer references and suffered a major PR problem. As an aside, this is interesting evidence that marketing departments cannot control the message; consider the Cluetrain theses.
  
• Hannaford's auditor has not been disclosed, but they've got some explaining to do. Their contract probably mandates that they may not publicly disclose who their auditor is, in anticipation of events just like this.

Impact on PCI:

• Organized crime has reportedly been shifting their focus from data at rest to data in transit as companies stop storing full track data and encrypt data on disk.
• This was the first significant theft of data in transit (sniffed off the network versus read off the hard drive). Note that network-level encryption would probably not have prevented this breach. Message-level encryption from the PIN pad or register might have prevented it.
  
• This was the first significant theft from a PCI-compliant merchant, calling the adequacy of the standards into question.
• Impact on PCI standards: they will likely require encryption for data in transit even on private networks. This will impact nearly every merchant and every payment-processing service provider.

Related News:

• Monday, March 17, 2008
• Tuesday, March 18, 2008
• Wednesday, March 19, 2008
• Thursday, March 20, 2008
• Friday, March 27, 2008

Fixing the Problem

I think the entire payment processing system needs to be reevaluated. We need to redefine the problem. The old problem statement might be expressed as "all the points in the transaction chain of custody need to be secured". A better problem statement might be "it's too easy to charge something to someone's financial account; all you need is a small amount of information". Security professionals derisively call this "security by obscurity".

Instituting PIN numbers on credit cards like we have on debit cards might be a reasonable first step. It would be relatively simple compared to other alternatives, and would take care of some of the fraud. Note that VISA's PayWave is a step in the wrong direction, particularly when a PIN or some other authentication factor is not required.

A better solution would be to build a new payment processing network with a three-party architecture (Kerberos-like) that provides (1) positive multi-factor authentication of both merchant and customer identity, and (2) non-repudiation features (such as those provided by the AS2 protocol) so that neither party could say that a specific transaction did not take place. The fees could be much lower, because fraud and chargebacks could be virtually eliminated. The substantially lower transaction processing fees would provide incentive for merchants to upgrade their infrastructure to accommodate the new payment processing network. The new system would be very expensive to implement, but considering the cost of fraud and cost of upgrading the security of every point in the existing transaction processing network in the desperate attempt to protect the obscurity of a scrap of information, it would be worth it.

Update: on Friday, March 27, the Boston Globe released additional information on the nature of the security breach (malicious software installed on point-of-sale servers in each of the stores). I updated my notes in this post to include the new information.

The Four-Hour Work Week

I read Tim Ferriss' book "The Four-Hour Work Week" last year. I'm not sure why. Maybe it was the provocative title. I think it was popular at the time because it fits in the "easy-money/get-rich-quick" genre. That being said, the first couple sections were actually pretty good. My notes are provided below; perhaps it will save you the trouble of actually reading the book (although it was a pretty quick read... I think I read it over a weekend).

Definition:

1. Establish your ideal lifestyle, recognizing that interests and energy levels are cyclical.
2. Focus on your strengths and natural interests; outsource (or enjoy healthy interdependence) where you're weak.
3. Working on a non-stop treadmill for 40 years, waiting for an eventual retirement so that you can finally enjoy life, is a lousy plan.
4. Recognize that money isn't the answer to everything, and enjoying life can be a lot cheaper than you might think.

Elimination:

1. Apply the Pareto 80/20 rule to eliminate busywork and extraneous stuff from your life.
2. Apply Parkinson's law (tasks swell to fill the time allotted) by setting short deadlines for important tasks.
3. Identify the one or two critical things that need to be done that day before you start the day.
4. Do not multi-task; apply yourself to one thing at a time and avoid distractions.
5. Batch up things like email and errands and do them once or twice a day.
6. Give up television, reading the news, etc. unless it relates actively to your current life focus.
   

Automation:

1. You can outsource just about anything! Consider a Virtual Assistant in India.
2. Learn to direct the work of others well (clear priorities, no busywork, clear instructions, time limits, etc.).
3. Create an income-generating asset that does not require your daily presence or involvement.
4. Tim Ferriss likes to sell a specific niche product in the $50-200 price range. It can be engineered/created, licensed, or simply resold. He provides a lot of tips on how to set up such a business.
   

Liberation (mobility):

1. Teleworking: Productive employees can negotiate for a remote working arrangement, where their productivity is measured with something other than hours in a cubicle.
2. Leave a bad situation: In some cases, you may need to leave your company to create a better situation.
3. Mini-retirements: take months (or even a year) off every now and then as a mini-retirement (sabbatical?) instead of a vacation.
4. Finding meaning: adding life (filling the void) after subtracting work

Beautiful Evidence

On January 30, I attended a one-day seminar put on by Edward Tufte on Presenting Data and Information. The course was fantastic, and for your fee, you also get a copy of each of his four books. They're magnificent, and they incorporate the principles that he teaches. I've since read Beautiful Evidence, the fourth book in his series. Here are a few highlights that stood out to me...

He makes the point that the principles of analytical design are derived from the principles of analytical thinking:

• Comparisons: show comparisons, contrasts, and differences.
• Causality: show causality, mechanism, explanation, and systematic structure.
• Multivariate analysis: show multivariate data; that is, show more than 1 or 2 variables.
• Integration of evidence: completely integrate words, numbers, images, and diagrams.
• Documentation: thoroughly describe the evidence. Provide a detailed title, indicate the authors and sponsors, document the data sources, show complete measurement scales, and point out relevant issues.
• Content counts most of all: analytical presentations ultimately stand or fall depending on the quality, relevance, and integrity of their content.

A good presentation is based on good analytical thinking. If your content is boring, then get better content! Focusing on fancy formatting and chart junk won't rescue your presentation. Have you ever worked with someone trying to create a presentation, and they use PowerPoint as a tool to think through the issues and craft the message that they want to present? It can be incredibly frustrating trying to work your way out of the cognitive fog. The tool actually hinders good analytical thinking. The cognitive style of PowerPoint includes:

• Foreshortening of evidence and thought;
• Low spatial resolution;
• Intensely hierarchical single-path structure;
• Breaking up narratives and data into slides and minimal fragments;
• Rapid temporal sequencing of thin information rather than focused spatial analysis;
• Conspicuous chart junk and PowerPoint fluff;
• Branding of slides with logotypes;
• Preoccupation with format over content;
• Incompetent designs for data graphics and tables; and
  
• Smirky commercialism that turns information into a sales pitch and presenters into marketeers.

If your presentation is unclear, then you should add more information. This is counter to the popular impression that if you need to clarify something for an executive, then you should "simplify" it by removing detail. But if you think about it, when you're looking at a photo, it's more clear when there is greater resolution, greater information density, and more visual information.

Even better than PowerPoint is a single 11x17" sheet of paper that presents a story with blocks of text, related graphics, and thoughtful analysis, incorporating the six principles of analytical design above. If you provide this as a handout, and people look it over before the meeting starts, then they'll be able to absorb the information much more rapidly than you can speak. Then you can use your time together for more productive discussion.

One more great quote from Edward Tufte:

"Making a presentation is a moral act as well as an intellectual activity. The use of corrupt manipulations and blatant rhetorical ploys in a report or presentation -- outright lying, flagwaving, personal attacks, setting up phony alternatives, misdirection, jargon-mongering, evading key issues, feigning disinterested objectivity, willful misunderstanding of other points of view -- suggests that the presenter lacks both credibility and evidence."

Making a presentation is a moral activity. Think about all the political speeches ("spin"), advertising blurbs, and marketing pitches you've heard, where corrupt maneuvers are epidemic. The same thing happens in a conference rooms every day, where people want to influence (control) your thoughts and actions. I value truth, authenticity, and clearer understanding of complex realities; I need to think more about my own presentations.

Blocking Advertisements

Speaking of spam, I also have a pet peeve against intrusive and obstructive advertisements. Blocking them in your web browser is easy. Firefox has great add-ons for this: Adblock Plus or Adblock, and Filterset.G. For Safari on the Mac, I find Safari Adblock to be simple and effective. It's a lot easier to focus on the content when you don't have all the dancing baloney on the page. Sometimes I catch myself thinking, "that's a strange web page design!"; then I remember that the block of white space must have contained an advertisement that I really don't need.

For blocking advertisements on the television, it's nice to have a digital video recorder (DVR) with a 30-second skip button. Even better is MythTV with a facility for recognizing advertisements and not recording them in the first place. On that note, the LinuxMCE distribution can make it easy to get a Myth system up and running. If you like old television shows, nothing beats watching five years of shows in series on DVD. I'm currently enjoying Northern Exposure via Netflix.

Spam

Spam (unsolicited commercial advertisements) comes to us in a lot of different ways, including:

• Email;
• Junk mail, delivered by the US Postal Service; and
  
• Telephone calls.

Email: I get about 100 spam messages/day, which Google helpfully filters out for me. At work, our top spam recipient would have received an average of 934 spam/day last month if we weren't blocking them. This would put the typical Monday morning inbox total at about 2500 spam messages! Or imagine getting a spam message on your Blackberry every minute and a half? It would constitute a denial-of-service attack. Happily, we have tools that can filter out these unwanted messages.

Junk Mail: I've come to think of the US Postal Service as a deliberate spammer. They don't offer a filtering service, and even worse, they solicit bulk mail business as a primary source of revenue. During the month of October last year, I measured our mail feed every day to see how much postal spam we get. I separated our mail into "desired" mail and unsolicited commercial mail. I counted the number of pieces and weighed each stack on a postal scale. By the end of the month, we had received 149 pieces of postal spam, weighing 18.3 pounds. It constituted 69% of our mail by count, and 68% by weight. The percentages were lower than I expected -- probably because we receive a good number of magazines, and we subscribe to Netflix. At this rate, we receive about 220 lbs/year of postal spam. One possible source of relief might be Earth Class Mail, but it's effectively a transitional technology as the need for the US Postal Service goes away. Magazines can be replaced by the web. Netflix DVDs can be replaced by video downloads on demand. Bills can be delivered electronically. Checks can be replaced by alternative funds transfer methods. Cards and letters are infrequent in the days of email. It's hard to think of a need for mail other than product delivery, which of course can be handled by UPS, FedEx, etc. As the US Postal Service's spam percentage climbs, we may reach a point where we simply decide to stop retrieving the mail. Let 'em pack the box with spam!

Telephone: we don't get too many telemarketing calls at home, because we have an unlisted number and we use a TeleZapper. Also, we have caller ID, and we can let unlisted or unknown numbers go to the answering machine. Personal calls to me usually come on my cell phone (I don't need the land line). At work, we have caller ID, and I answer my phone less than 10 percent of the time. I get a lot of unsolicited sales calls (10 calls/day?) from people trying to sell expensive enterprise hardware, software, and security solutions. Of course, one of the reasons they're expensive is because they have to pay sales people to make cold calls, schmooze potential clients, and so on. More on this topic later; meanwhile, I just let most of my calls go to the answering machine, and I feel no obligation to return unsolicited sales calls.

Of all the sources of spam that I get these days, the US Postal Service is the worst.

Annual Deep Think

From Calistoga 2008

Last weekend, I joined a few friends for our "Annual Deep Think" in Calistoga at a very simple resort (Mountain Home Ranch). We've been doing this for ten years now. The objective is to get some extended time alone with God, apart from the distractions of regular life, in the company of friends. There's virtually no "program", and the keynote speaker is God. I usually come with a few questions on my mind. I think about big-picture stuff, annual goals, and so on. I find a nice perch in the woods, along a stream, or on the edge of a cliff somewhere. I read the Bible. I talk to God. I talk to my friends. I try to listen :)

It's an awesome experience! The natural setting really helps me "get in the zone." I'd share a few questions or themes with you, but they're not exactly ready for print. However, if you'd like to join us next year, then you could engage in the process and you would hear a lot of good questions and stories!