LostMailIn the legal industry, we tend to do quality by proxy. If you went to a high prestige brand-name law school, many assume you received a better legal education than if you went to a law school with a less prestigious brand. We make the same type of assumption about law firms. If you work at a high prestige brand-name law firm, many assume the quality of your work is better than if you work at a firm with a less prestigious brand. We don’t have evidence to support these assumptions, but they have been part of the legal industry for many decades. They even persist when there is evidence to the contrary. Consumers may be reducing their loyalty to brands, but lawyers still place a lot of trust in brands.

Trusting the brand, versus trusting metrics that measure desired characteristics, is measuring something by proxy. Our clients do this when they evaluate our legal work. Clients generally don’t know a good quality legal brief or contract from a poor quality product. They do feel qualified to measure other things, though. They measure the things they feel qualified to assess, and use that measurement as the proxy for the quality of our legal work. This measuring by proxy can be especially important when clients have direct access to your work environment.

When I was a general counsel, I would coach lawyers new to the in-house world about key differences between the law firm environment and working in a corporation. For example, today most clients never see a lawyer’s office if she works in a law firm. The clients never get past the conference room. But, in a company, clients wander in and out of your office all the time. For many clients, especially senior executives, an out of control office can mean an out of control lawyer. If they see a mess, they assume your thinking is messy. Quality by proxy.

Whatever the reasons for the quality by proxy approach, it is time for the legal industry to move away from that view and into measuring quality directly. A fancy law degree, nice offices, and an impressive pedigree should not serve as a proxy for, in law firms or law departments, superior quality services.

Arguing for Quality Legal Services

Ron Dolin recently published a nice analysis of quality measures in the legal industry. Dolin is, among many other things, a Research Fellow at Stanford Law School. His conclusion, “Standardized quality metrics are front and center [in the legal industry].” I also have published posts (e.g., here, here, here, and here) arguing that quality measures are possible and should be used in the legal industry.

Dolin says, “Efficiency in legal services without quality is useless.” This is a basic idea underlying lean thinking, which is one of the leading operational excellence philosophies. Since you may not be a follower of operational excellence, let me explain the efficiency-quality connection a bit further.

When we do poor quality work, we usually need to fix the work. We may need to correct errors in the work or we may need to fix whatever happened as a consequence of the poor quality work. To get past poor quality, we start by defining quality.

In lean, quality is defined by the customer’s needs. Customer means the person who receives your work and either does the next value added task or is the end user. The customer is the next person in the value chain. Assume you are an associate working in a law firm. You are asked to draft a contract and give it to a partner for review when you are finished. We define quality by what the partner needs from you to perform her review. The partner is your customer. If the partner expects a contract finished in every respect and ready to send to the client, then something less means you delivered less than 100% quality to your customer. The partner will either fix your poor quality work or return it to you and have you bring the poor quality work up to snuff. Either way, we have introduced waste into the process in the form of rework. In the legal industry, rework is common. That is one of the reasons legal service delivery is so inefficient.

Of course, waste from poor quality work often gets built into the chain in many spots. The partner may hand over to the client a contract that is not at 100% quality. If you think this doesn’t happen, then think about the stories clients tell of getting work product with the names of the company or employees wrong. Poor quality, of course, goes deeper. A lawyer may get the story wrong, leave blanks to fill in missing facts, or make changes without consulting the client which then have to be reversed. I saw all of these happen many times when I was in-house counsel.

The point of Dolin’s post, however, was that we need to get on with it and introduce standardized measures so we can improve quality (and improve efficiency). Although lawyers may protest, we can introduce quality measures relatively easily.

Quality in Contracting

I’ll walk through a few quality measures that are easy to introduce when working with documents. These aren’t the only quality measures and I’m not even going to argue they are the best. These are a starting point. If you want to put teeth in the measures, tie performance to compensation.

Readability. Contracts are, or at least should be, written so that people can read and understand them. We will reach a time when many contracts will not require human intervention (so-called Smart Contracts), but for now, we need people in the loop. However, if you look at most contracts written today, they are unreadable. To change this, establish readability standards for the contract. Lawyers shudder, because re-writing the language of a contract means using language a court may not have seen. On the other hand, if a contract is written using simple, clear language it is unlikely a court will need to see it, and if it does it is hard to argue confusing language will improve your case. There are many freely available methods for scoring the readability of texts (I frequently use the Flesch-Kincaid method which scores readability according to school grade level). Set a standard (e.g. 8th grade) and require that all written contracts meet or beat the standard.

Accuracy. It seems obvious. A contract should be accurate. Names should be spelled correctly. The contract should not have any typos. Numbers (dates, prices) should be correct. Internal references in the contract (e.g., see paragraph 4) should be accurate. Yet, those of us who have handled many contracts over our careers can say that seldom do we see contracts that meet this obvious test. It is easy to pick up contracts, and I include in this contracts that have been poured over by dozens of lawyers such as financing contracts for more than $1 billion, and find errors. Lawyers seem to feel this is just the way it is. After all, we are human not perfect. Our eyes get tired, we get bored, and we miss things. The cost of perfection is too high, so the goal is to avoid mistakes on things that count and assume that errors on other things will be worked out if and when there is an issue. But of course, those are excuses. There are many ways to reduce errors. For example, simply teaching people how to use software would help (automatic numbering and cross-reference updating). If you want change, set a standard. Put all these issues in a bucket and start with a standard that defines the maximum number of permissible errors. Below that, and the drafter’s score falls below 100%. Use that approach for each step in the drafting process (and no, you shouldn’t pay more to get the better quality, you should pay less for quality below 100%), and watch how fast quality improves.

Custom. Let your mind roam free. If you get drafts of contracts later than the agreed date, then set a standard for what “complete” means and the date by which you must get the complete contract. Anything less, and the score drops below 100%. Don’t get fully signed contracts back on time? Set a standard – all contracts must be signed and returned to the law department within two weeks of the contract being delivered to the signatories. Anything over two weeks and the score drops below 100%. Wait, you say, I don’t control when the other party signs. No you don’t. But, if there is a standard, if we score against the standard, and if we start tying something meaningful to that score (performance review, compensation), then you will see ingenuity kick in and performance will improve. Just as Dolin said, “efficiency in legal services without quality is useless,” we can say “measurements without consequences are useless.” If you want behavior change, there must be a consequence for not changing behavior.

Quality Legal Services Should be Mandatory

The quality metrics I discuss above are just the beginning. We can add other metrics, such as whether a contract results in a dispute, the severity of the dispute, the cost of a contract over its lifecycle, the number of times a contract must go back and forth between drafters before completion, and so on. Your choice of quality metric should relate to where you see issues in the value chain for the contracts. Your quality metrics also should evolve over time. If you reach a point where every contract scores 100% on the accuracy measure, then it should recede into the background (meaning you might not stop measuring it, but it no longer is the primary measure) and you focus on another aspect of quality with a new measure. As you gather more data, you can bring more sophisticated metrics to bear, which may help you dig deeper into the root causes for quality issues.

It can be easy today to forget what quality means in our ever day lives. We have all heard about the Six Sigma quality measure (in statistics, six sigma means 3.4 errors out of every 1,000,000 instances). Another common measuring point is Three Sigma (or 66,800 errors out of every 1,000,000 instances). Think about my comment that we can find errors in most contracts, and compare that quality performance to the following metrics for common things at both the Three and Six Sigma levels:


  Three Sigma Six Sigma
Lost Mail 20,000 letters per hour 7 letters per hour
Unsafe Drinking Water 15 minutes per day 1 minute every 7 months
Incorrect Surgical Operations 5,000 operations per week 1.7 operations per week
Short or Long Airplane Landings 2 per day at most major airports 1 every five years at most major airports
Incorrectly Filled Drug Prescriptions 200,000 per year 68 per year
No Electrical Service 7 hours per month 1 hour every 34 years


Dolin says, “measuring quality is key in the modernization of legal departments, as well as their legal services vendors and/or law firms.” I think it is more than that. I think that lawyers are fighting a battle right now, albeit one most seem to believe isn’t happening, to show why we remain relevant to our clients. Entrepreneurs, technologists, and others without law degrees (and some with law degrees) see opportunities to provide our clients with higher quality services, more efficiently, at lower prices. They may not take away all that we do, but they sure can take away a lot of what we do. Perhaps that is the way it should be. But, if you are a lawyer who wants to remain relevant and provide superior client service to her clients, then I suggest you make quality something mandatory on your list and you put some teeth into your program.