How many of you have written an “open book” exam?
An open book exam is an exam you write while you sit in a library withe access to all the information in the world.
Can’t be tough, right?
Every time my class had to vote on the format, we rejected open book exams.
Why?
Doing well in open book exams actually simulates performance in real life situations, i.e., the application of existing knowledge to cutting edge problems in new ways.
Far more difficult that regurgitating the same old - repeating known ideas to score marks.
An open book exam means having to do the hard work. Of actually understanding and mastering a subject - reading, of understanding and then being able to produce something useful.
Remember when they stopped coming to your law offices for the books?
15 years ago is when we first saw the change from clients coming to visit your office because it contained a law library.
Specialised knowledge being the preserve of specialised places stopped being a thing when Google and the Web democratised access to ideas.
Did it make legal research easier? Yes and no.
Did it make specialised legal research less important? No.
Research is as important as ever.
Quality research and analysis is as rare as it ever was, as valued as it ever will be.
Reading is an underrated skill
If all the world’s databases and knowledge materials can be found on the internet, what distinguishes one professional from another?
Photo by Anna Dickson on Unsplash
Reviewing a draft, a petition, a research paper or an argument to generate further value requires a trained and skilled ‘reading eye’.
Eyes that clients continue to value an pay top $ for.
AI makes this work more important.
Much of what professionals previously read were prepared by other professionals. Sometimes less skilled than them, necessitating their expert eyes add a layer of critique and expertise to improve the product.
Also, much of professional work involves generating at the bottom of the hierarchy with review across multiple layers of horizontal and vertical expertise.
Now we see that some of the generation at the bottom is being done by AI.
The work of the “reviewers” hasn’t changed.
In fact, if you are at the bottom of the system generating the first drafts, you have moved up the chain, if AI is doing the generating for us.
You are now the first layer of ‘review’.
The first pair of expert eyes.
AI doesn’t make your work dangerous if your work wasn’t dangerous before.
One reason why I find the fears of AI overstated or formed on a gross misunderstanding of what AI is capable of, and not.
If you were doing anything important, that work is subject to layers of review. Human error is as dangerous as those made by an AI system, and where the stakes are high, organisations/professionals are set up for review processes that take care of errors.
If it wasn’t important enough for levels of review, then it can’t be dangerous now merely because AI is involved.
What if court judgements or legal opinions are written by AI?
Photo by Unseen Studio on Unsplash
This is bad framing.
Because important opinions and judgements aren’t written by one person. They are produced through a process of generation, iteration and review.
In the past, the initial act of generating a draft would occur by a paralegal, intern or junior professional.
If a judge or lawyer were sending this unreviewed for publishing to their clients or in the form of a judicial pronouncement, then we would have had the same problems as, say, if today they were to push what ChatGPT generated onward without review.
Is AI more dangerous or less competent than a paralegal or intern?
Maybe, maybe not.
The point is never going to be how reliable or not AI is, but how reliable the people are who are supposed to be doing the actual work.
AI can be a powerful tool in the hands of those who know how to use it to improve their work.
It can also be a fantastic but inexperienced intern when led by a boss who is lazy and puts out unsupervised work.
Some of the debates on this really need to get less unhinged.
Meanwhile, those who can read carefully and expertly are going to be that much more important in a world of generative AI.
The discerning eye has never been more valuable.
P.S in public interest: When you use TipsyTom to generate a first draft, review every word before you send it out.