Security

What Does ‘SHA1 is Broken’ Mean?

Eric Marvets — Wed, 12 Dec 2007 07:35:00 GMT

For those of you Slashdot readers out there, you may have heard by now that ‘SHA1 is broken’. Recently I did some security videos for Microsoft, and decided that SHA1 was the best hash function for the example (modifying an existing application to store hashed passwords).

The videos I did were part of the “How Do I” series, and not exactly the place to explain why it was appropriate to use SHA1. But for those of you looking to understand the why behind the example, I’ll take a few minutes to explain it.

What exactly is SHA1?

SHA1 is a hashing algorithm, also known as a one way function. A one way function is where given any value of x, it is easy to find f(x), but given f(x) it is unrealistic to find x. One way functions allow us to take a ‘fingerprint’ of data without storing the data itself. In a password scheme, instead of storing a user’s password (x) we instead store a hash of the password (f(x)). Later when the user wants to login, he again supplies a password which we hash and compare against our stored value.

It’s also useful for ensuring the integrity of data. When a message is sent over an unsecured channel, a hash of the message can also be used to check the message once it reaches its destination. If the message does not match the hash, then we assume it was modified in transit.

Designed Strength of SHA1

When we hash data, the range of values for x is infinite. The hash on the other hand is a fixed size. Therefore, for each value in the range of our hash, there are an infinite number of possible values for x.

This range of possible values determines the odds of guessing a value x to match a known value f(x). If the size of the hash value was 2¹, there would be a 50/50 chance that the valued guessed would match our known f(x). That’s why SHA1 utilizes a very large hash size of 2160. To put that in perspective, the Earth is composed of 2170 atoms. It’s computationally unrealistic that anyone would be able to beat those one in 2160 odds to find a value x which matches our known value f(x) (with today’s technology).

The Birthday Paradox

Some of you may be asking yourself, “but I read on Wikipedia that SHA1 has a strength of 280?” This is true, but to understand why, we will first look at the birthday paradox.

How many people must be in a room before the odds are even that one of them shares your birthday?

How many people must be in a room before the odds are even that two of them share the same birthday?

In the first question, we are looking to match a specific value, while in the second we were just looking for any 2 matches. The answers are 253 and 23. The reason for the difference is that between the 23 people, there are 253 unique combinations. In one way functions, this is the difference between finding what we call a pre-image value versus a collision.

The reason we say the strength of SHA1 is 280, is because we are talking about finding collisions (any two values for x with the same f(x)). When we are hashing passwords, we are asking the person logging in to match a specific f(x), and the strength of SHA1 in that situation would be 2160.

The Current Strength of SHA1

The analysis of SHA1 shows that collisions were found in 263. It’s now becoming computationally feasible to find two values of x that match an f(x). It’s still short of being probable that those two matches found would allow an attacker to compromise an encryption system, but the worry is that SHA1’s strength will continue to decline.

Until the strength of SHA1 drops to 240, it is still a valid way to protect against pre-image attacks.

Why Did I Choose SHA1?

In addition to SHA1 being secure in the example, there were a couple of other reasons I choose to use it instead of something like SHA256 (2256). The first reason was that in the example, I was showing how to modify an existing application, by simply changing the value in the password field from a password to the base64 string representation of the hash, which is 28 characters in length (for SHA256, it would be 44 characters). If the database allowed passwords that size, then it’s trivial to add support for hashing.

The other reason is that there are far easier ways of attacking a password field than targeting SHA1. An offline dictionary attack against the users’ passwords is several orders of magnitude easier. SHA1 protects the hash against brute force attacks. It does nothing to protect a user who chooses a poor password.

A system is only as strong as its weakest link.

-Eric Marvets

DRM Scorecard Makes Me Wonder: The Media Industry and the TSA, Sadistic or Incompetent?

Eric Marvets — Thu, 02 Aug 2007 08:19:00 GMT

Back in March, I posted about the media industry and the BORA principle, or break once, run anywhere. Info week has a DRM scorecard where the box score reads Hackers 1000, Industry 0.

This all goes back to the simple fact that all DRM is based on encryption, and that it’s illogical to give someone the decryption key that is required to enable what the media industry views as authorized behavior (media playback) without expecting someone else to utilize that decryption key for other behavior, such as making Fair Use backups or sharing it on a P2P network.

Encryption is defined as the science and study of secret writing. What is it that the media industry is trying to keep secret? While we may want I Now Pronounce You Chuck and Larry and Who’s Your Caddy to be some sort of secret internal referendum on the crap the entertainment industry regularly produces, we have to assume from their actions (theater release inevitably followed by mass DVD production) that they are proud of their works and wish to share them with the entire world.

They worried about piracy with VHS, and it turns out that may have in fact saved Disney and launched an entire consumer market for home video. They worried about it with DVD’s, which have brought in billions of dollars to the media industry despite the fact that CSS was broken in 1999. Their fear and illogical behavior impedes and irritates their consumers while having absolutely no effect on the spread of piracy (which they could easily defeat should they ever focus on the simple economics and technology of the pirating industry).

I would be happier if the media industry and the TSA were sadistic rather than incompetent. It would be comical to see these two groups meeting for the first time over drinks trying to one up each other:

“We made a list comprised of thousands of names. If you fly and your name is even remotely similar to one on the list, we do extra searches…every time you fly….over and over again. The kicker is we let anyone with Photoshop and a printer board under any name they want.”

“Oh yeah, well we sell malleable $.05 pieces of plastic for $20 and when it gets scratched or stolen, we force them to buy a new one because we don’t allow them to make backups. Even though anyone with technical skillz can download the same thing for free.”

“Oh yeah, well we found a way to make people who can’t even change at the gym without flip flops walk around barefoot in public. We tell them we’re screening for bombs and they just go with it. The terrorist can still strap whatever they need to their leg, just not their shoes.”

“We installed rootkits on people’s PC without their knowledge.”

“We banned water and baby food.”

“We sue the people who love our products the most.”

“We detain babies.”

“We…damn you! Stop playing the baby card, that’s not fair!”

Encryption Presentation - .NET Developers Group - NYC Microsoft Offices - June 21st

Eric Marvets — Mon, 02 Apr 2007 06:46:00 GMT

For those of you in NYC or the surrounding area, I will be doing a presentation on encryption at the .NET Developers Group on Thursday, June 21, 2007. It’s a similar presentation to the one I’ve done for a number of user groups in the Southeast. I made this presentation as a response to the flood of online code snippets for encrypting data. While they are all fairly easy to use, they don’t explain what they do and often developers think their data is more secure than it actually is.

During the presentation, we’ll quickly cover some high level encryption basics (asymmetric, symmetric, and one way hashes), but will spend most of our time dealing with symmetric encryption; namely how and why you configure a symmetric algorithm to encrypt the data (ECB vs. CBC). By the end of the session, you’ll finally understand what an initialization vector (IV) is used for and the proper way to create and store it.

Don’t worry if you don’t understand what half of that meant. I’ll be sure to explain everything as we go along.

You can also find a fair amount of the content from the presentation here in an article I wrote a while back.

When Will the Media Industry Embrace the BORA Principle?

Eric Marvets — Thu, 22 Mar 2007 10:06:00 GMT

I was reading the WSJ this morning and came across an op-ed piece entitled “Congress Must Make Clear Copyright Laws to Protect Consumers” written by Walter S. Mossberg. I enjoyed the article and especially liked his fair use comparison between print and video (you can reprint a small section of a publication in another without permission, yet you can’t post a short clip of the “Daily Show” on YouTube). The one issue I had with his article was referring to Apple’s FairPlay as a “DRM system for music that has worked” (it’s not the DRM, but rather the void in the marketplace that made Apple successful).

While I would love for Congress to fix our copyright laws, I regard the notion as fantasy. They don’t appear capable of fixing any complicated issue and tend to muddy the waters making any situation worse off than when they began. Secondly, the media industry will either collapse under the weight of their archaic business model or realize the impossibility of DRM and move in another direction. Either of which nullifies the issue.

DRM is impossible due to the fact that it falls under the BORA (break once run anywhere) principle. This principle is understood thoroughly by those of us in the security industry. When analyzing a threat, if it’s determined that an entity could be compromised once and then be exploited globally, you are faced with two choices: restrict access to the entity by limiting and hardening access points or decrease the exploitability of the entity once compromised.

Many industries have fought BORA, which is akin to fighting gravity. I can think of three this morning, namely the software, credit card, and media industries. It’s infuriating to think of all the revenue lost and the exorbitant externalities bore by an unassuming public all because these industries couldn’t understand simple logic. This is especially true when the solution requires only a trivial leap of faith.

The credit card industry is by far the clearest example of an industry that came to terms with the BORA principle. Quite frankly, they delayed the success of ecommerce by about 5 years. I’ll even go so far as to say that we would not have had a dotcom bubble if not for their foolishness.

In 1992, credit card fraud was at its peak (15.7 cents per $100 charged) due to fraudsters becoming more advanced. The internet allowed people with similar interests who would have never came into contact in the physical world to find one another digitally. Fraudsters were able to share information and increase the sophistication of scams long before e-commerce was a reality.

Faced with a bleak economic picture, the credit card industry became paralyzed by fear as they imagined credit card numbers floating unprotected through cyber space. For 6 years their agenda was to spread fear in the hopes consumers (and brick and mortar retailers) wouldn’t embrace ecommerce until they created a process by which credit card numbers couldn’t be stolen online.

Their fear clouded their ability to approach the problem logically. If a credit card can easily be cloned by your waiter at a restaurant, then why protect the same card during an online transaction? Or better yet, why protect individual transactions while every brick and mortar retailer has a record of each credit card used for purchases? As a criminal you target the warehouse, the delivery truck, the retailer, but never a single customer.

The history of ecommerce between 1992 through 1997 is fairly interesting and comical. The failure to realize what seems obvious today is not the fault of a single company. There were over 30 dotcom companies that were created during this period, all vying to be the payment processor for not only the web, but literally the future. In 1994, Visa and MasterCard turned to Microsoft and Netscape, respectively, for solutions. As any company would, these tech giants devised schemes that benefited them rather than serve the needs of their clients.

Fortunately for Visa and MasterCard, CNP (card not present) transactions were already allowed for mail order catalog purchases. Despite their fear campaign and merchant agreements that left stores 100% liable for fraud, companies like Amazon accepted the increased risk and allowed the credit card industry to ultimately be successful. By 1998, Visa’s sales volume had tripled which cut fraud as a percentage nearly in half.

Credit cards went from being used for credit to being used for convenience (what they were originally designed for in the 1950’s when the banking system was fractured). This was a massive shift in the financial industry. Comparing one’s own experiences in the checkout line at a grocery store in 1992 and 2002 tells the story. It went from checks and cash to plastic. Even the stigma of credit cards is completely different today. College students can’t survive without credit cards, a far cry from when they were counseled not to have one.

With this shift, credit card companies began focusing on preventing fraudulent transactions. By using two sets of data, one for CNP and the other for in-store transactions, they were able to prevent cards that were cloned from being used on the web, and card numbers stolen on the web from being used in person. The other advent was address verification, which among other things allowed retailers (who are liable for fraud) to prevent highly liquid assets from being shipped to any address other than where the statements are delivered.

They then began to promote ecommerce as if they never said anything bad about it in the first place. Consumers were given zero fraud guarantees which created a perception of little to no risk. It wasn’t long before traditional brick and mortar retailers rushed to the web, displacing overnight dotcom sensations which lacked feasible business models. Finally in 2003, we were at point that could have been accomplished in 1998.

There are many parallels between what the credit card industry went through and where the media industry finds itself today. Instead of focusing on preventing the Fair Use of their content, they should instead deliver it through open mediums creating additional revenue streams while increasing the popularity of their product. Piracy can be handily defeated, not through the legal system but rather through a firm understanding of the economics of the environment.

Today, credit card companies are at the peak of their success. In 2004, the fraud rate for credit cards dropped to an all time low of 4.7 cents per $100, while setting records for volume and profits. I know for a fact the same thing can be accomplished in the media industry because I’ve studied it. All it will take is a trivial leap of faith.

-Eric Marvets

Rampant FBI Abuse of Power – Now Paperwork Free

Eric Marvets — Tue, 20 Mar 2007 23:53:00 GMT

Last week I read on Schneier’s blog about the rampant abuses of new powers granted under the Patriot Act. Today I read over on the Washington Post that the new rules from the FBI’s general counsel directs agents to use better discretion and to ditch the paperwork.

Wasn’t it the improper paperwork that led us to discover the FBI improperly obtained data? How does instructing agents to limit requests to the most dire situations, to not file follow-up paperwork (grand jury subpoena or national security letter), and to ask for the data orally fix the problem?

Here is the new process:

FBI suspects you of a crime.
FBI calls the phone/ISP/bank/etc. and asks for data under the emergency provision.
The company takes their word for it and provides the data.
FBI doesn’t have to obtain warrant and receives data without a single sheet of paperwork.

Anyone see how this could be abused? Forget judicial oversight, now they don’t even have agency oversight.

Sickening…

Month Of MySpace Bugs

Eric Marvets — Tue, 20 Mar 2007 22:46:00 GMT

I just found this little project called Month of MySpace Bugs. This should be interesting to keep an eye on. As they state, they are only picking on MySpace (they could have found similar problems in any of the poorly crafted social networking sites) because they are trying to get attention, MySpace is extremely popular to get them even more attention, and that MySpace is “notoriously dickish” in response to security issues.

Starting on April 1, they will release one MySpace hack a day. Most will center on XSS attacks and they invite anyone to send in a hack as long as you have a proof of concept. It sounds pretty light hearted and looks to be half goof, half public beating.

Google Changes Privacy Policy

Eric Marvets — Thu, 15 Mar 2007 08:31:00 GMT

I personally love Google and have since the first search I made 7+ years ago. I remember it clearly, I was at my first dot com gig and I made a search for some obscure technical detail. I was shocked that the first result took me to what I was looking for, and it has been my homepage ever since.

Unfortunately I’ve had to turn a blind eye to the serious privacy concerns I’ve had. The functionality they provide is so critical to my job that I have chosen to slightly alter my behavior and then proceed with indifference.

I always configure my browser to not store cookies (the number one feature to choose FireFox over IE); a practice that was instigated when I learned Google sent the same cookie from each machine when a search was made. I use Tor and Privoxy to achieve true anonymity when researching certain topics. I will never use their desktop client.

However, two pieces of good news have come out of the Googleplex in the last few months and I hope this is the start of a trend. The first story appeared mid January when it was reported that Google was resisting certain government subpoenas. With every financial transaction in the US being one step away from public record, phone companies granting the NSA unfettered access to their data, and ISPs supplying information to any ‘legitimate’ group that requests it, this is a breath of fresh air.

The second item came out yesterday on Google’s corporate blog. They announced a new log retention policy that would anonymize the data after an 18 to 24 months period unless they are legally required to retain them. They also said they would look at improving users’ privacy across the board, including services like Google Chat and Google Desktop.

This announcement was a change in corporate policy, rather than a detailed technical plan. We can probably expect the specifics in the near future. Their current stated intention is to change parts of the IP address and the cookie. Unless they completely strip the logs of the IP and cookie, then it will never truly be anonymous, but I think they will change it to the point that the data could never be used in a court of law.

It’s not perfect, but it’s better than nothing. They weren’t facing any possible government sanctions for eroding users’ privacy; in fact the exact opposite is true. They did this on their own accord. I don’t know whether to praise or expect this kind of behavior from a company whose motto is “Don’t be evil”, but for today, I say praise.

The world will be a completely different place 10 to 20 years from now. I am always infuriated by people who claim they have nothing to hide, so they have no privacy concerns. Privacy is a vital element of the human condition. I hate to imagine the political spectrum in 20 years. If they can find a drunk driving arrest from the 70’s now, what will they be able to dig up with your search, email, phone, and financial data at the fingertips of the established power base. Just because we use this data now to optimize search engines or hunt for terrorists, does not preclude malicious usage in the future, on a scale we can not imagine…

non facias malum ut inde fiat bonum

-Eric Marvets

Really Good Essay on Privacy

Eric Marvets — Tue, 23 May 2006 11:43:00 GMT

I just read a really, really good argument for privacy:

Too many wrongly characterize the debate as "security versus privacy." The real choice is liberty versus control. Tyranny, whether it arises under threat of foreign physical attack or under constant domestic authoritative scrutiny, is still tyranny. Liberty requires security without intrusion, security plus privacy. Widespread police surveillance is the very definition of a police state. And that's why we should champion privacy even when we have nothing to hide.

I really would like to see a political party actually take a stand for liberty, but that is very unlikely. Even though 9/11 occurred almost 5 years ago, the political climate is still too hot. The legislation past in the short time after the attack by fear mongers and those looking to gain power from the tragedy of others still gains strength today.

Once we give up liberties we never get them back.

Preventing Fraud

Eric Marvets — Fri, 12 May 2006 19:01:00 GMT

Sometimes I think we app sec folks get way too bogged down in the finer points of CAS or protecting application service credentials and tend to stray away from protecting an application against fraud. Both are definitely important but we tend to focus less on the latter. One of the main reasons I respect Bruce Schneier (besides his obvious crypto ninja skillz) is because he is constantly advocating for greater consumer fraud protection.

What I am Trying To Accomplish:

I wish to create a process where a large set of text (addresses) could be compared to determine exact matches and close proximities to put into a fraud report. Searching for only exact matches would be easy, but a crafty attacker could alter the text slightly while still enabling a package to be delivered to the proper address, i.e. 123 Mulberry Lane, 123 Mulbarry Lane, and 123 Mulberry Ln. or Clearwater, FL 11111 and Claerwater, FL 11111. The goal would be to return each of these addresses as a near 100% match as possible.

I thought of developing a one way algorithm that performed exactly opposite of SHA, where a small change to the input created a minimal change to the output. The output could then be used with a second process to compare all these numbers to find near identical matches.

Real World Example Where This Would Be Useful:

Recently, I was given an interesting problem that involved preventing credit card fraud. To demonstrate the problem domain, let’s just say we are dealing with a site that allows users to purchase gold at a small percentage over current market rates with a credit card. This business faces a few interesting problems.

First and foremost it offers a very attractive vehicle for people to launder money. Using a stolen credit card to purchase an untraceable and incredibly liquid asset at 10% over its market value is a lot more attractive compared to buying high end electronics equipment with serial numbers and then selling it on eBay at a fraction of the purchase price.

Secondly, as the amount of fraud increases, so will the merchant rate on credit card transactions. If they are to sell gold at 10% over market, they definitely would want to get the percentage charged by the credit card companies as low as possible. If the amount of fraud becomes too great their merchant account will be revoked preventing them from accepting credit cards effectively putting them out of business altogether.

Ideas for Preventing Fraud:

The first thought I had was to develop an algorithm that effectively throttled the amount that can be purchased by customer AND credit card AND corresponding shipping address. The algorithm would use not only the length of time, but the number of previous valid transaction in determining the amount that can be purchased.

A crafty attacker would see that a way to side step the throttling algorithm would be to use multiple different customer accounts and credit cards to make a large number of small purchases. I had two ideas to combat this. First would be to compare the number of different credit cards used to ship an order to the same address. Second would be to compare the number of different accounts that placed orders from the same IP address.

I am also toying with the idea of comparing the location of the IP address used to make a purchase in relation to the billing address. I believe this may cause too many false positives and take away from the amount of time customer service reps spend analyzing the output from the other processes.

At the end of the day, the goal is to have a fraud report that presents each order placed sorted by a suspected fraud rating and detail the evidence that supports the rating. A fixed number of customer service reps would then spend their time validating as many orders as possible before they are actually shipped. Once they are out the door, all you can do is maintain what amounts to log files that you can turn over to the credit card companies in case someone makes a fraud complaint.

The one way hash seems to be the optimal route for comparing addresses. Anyone have any thoughts on how to accomplish this? Can anyone think of any other fraud protection algorithms for a business like this?

-Eric Marvets

"Geeksta" Rap

Eric Marvets — Wed, 10 May 2006 12:34:00 GMT

Rap for nerds?? Call it Geeksta rap or Nercore, it’s pretty funny. Check this one out about “Alice and Bob”. For those of you who don’t know, Alice and Bob are commonly used in crypto examples where they often interact with Trent the trusted authority and Mary, the malicious user.