Blog

Conceptual Search Using AI

The ability to search is what makes digitization such a powerful and pervasive force in the world. We can easily navigate electronic documents on our computer or tablet or phone. We know what simple search looks like, and we know what it does.

Everyone benefits from simple search, but arguably, nobody benefits more than legal professionals. Simple search is a great way to find a word or short phrase when reviewing legal documents. However, it is not the only way.

A much more powerful, more complex method of search exists. We call this form of search conceptual search, and it functions similar to simple search but with some incredible differences.

Simple Search

First let’s talk a bit more about what is happening behind the scenes of simple search. Open up Word, or Excel, or a browser page, and click the magnifying glass (or “CTRL+F”). Type in the word or phrase you want to find, and immediately see a list of each instance of that word or phrase in your document. The rest of the work is up to you, but simple search gives you a big lift early in the process.

Simple search is a powerful tool when you know what you’re looking for, and when what you’re looking for is straightforward. You already know the word or phrase, and the computer looks for that exact matching sequence of characters.

What about when you don’t know exactly what you’re looking for? Legal professionals know that sometimes finding the really important text in a contract requires more than just looking for one or two key words or phrases. Context is important when trying to discover more complex concepts within lengthy contracts. Thus, finding all of the relevant information in a contract or other legal document requires more sophisticated search.

This is where conceptual search comes in.

Conceptual Search

Even the most boilerplate legal contracts often contain differences and idiosyncrasies in their structure and phrasings. When compared to other contracts of the same type, even NDAs – arguably the most standardized type of legal contracts – are far from identical, one to the next. And what if you’re working on contracts from more than one jurisdiction? Or working with a set of contracts from multiple countries? Multiple languages?

Simple search just matches the characters you type into the search bar with identical strings of characters somewhere in the contract. Conceptual search can accomplish this, too. But it can also build a more useful web of word associations and context that can capture words and phrases that are otherwise distinct from the verbatim text of your search, yet vitally important to the tasks of compliance, diligence, discovery, review, negotiation, remediation, etc.

2020 is the year LIBOR remediation, and what comes next, is on everyone’s mind. And although the LIBOR problem is relatively clearly defined, discovering whether your contracts are LIBOR-exposed can become complicated. For example, the paragraph below – as well as other, similar paragraphs – is common within syndicated commercial loan agreements:

“Base Rate” means a fluctuating interest rate per annum in effect from time to time, which rate per annum shall at all times be equal to the highest of:

  1. the Prime Rate; and
  2. 1/2 of one percent per annum above the NYFRB Rate; and
  3. the LIBO Screen Rate for a one-month Interest Period.

In this text sample, we have two phrases, “base rate,” and “LIBO,” neither of which are “LIBOR”. A simple search wouldn’t find these words or phrases. Conceptual search, on the other hand, would find these terms and relate them to “LIBOR” in a quantitative way. What does that look like?

Word Embedding

ContraxSuite deploys conceptual search in order to find not just the word “LIBOR,” but the general concept of “LIBOR”. One of the major techniques to conduct conceptual search is word embedding. Word embedding programs like word2vec build language models that capture both exact term matches and near matches. A word2vec model is trained on hundreds or thousands of contracts that are all, for example, commercial loan agreements tied to LIBOR. Similarity is measured based on how often a word appears near a target word (“LIBOR”), how close it is (Two words away? Two sentences away?), and the context of surrounding words and phrases (e.g., do “LIBOR” and “base rate” both appear in the same sentence as “interest”?).

libor similarity search

A conceptual search for words that are analogous, or very similar, to “LIBOR”

In the image above, we can see that in a set of contracts, the word “eurodollar” has the closest relationship to the concept of “LIBOR”. We can infer that searching for contracts that don’t mention “LIBOR,” but that do mention “eurodollar,” might help us find contracts that are LIBOR-exposed even though they don’t necessarily mention “LIBOR”. Word embedding models can provide this kind of detailed knowledge of how each critical concept within a contract set relates to all the others, and can help legal professionals see a clearer picture of the full impact and meaning of their contracts.

Conclusion

Humans, especially legal professionals, rely on conceptual search all the time. Conceptual search is so rudimentary to the way our brains operate, that we often don’t see the difficulty in programming machines to conduct conceptual search. Luckily, there are software tools available that can utilize word embeddings and other conceptual search techniques that are similar to how human brains search for and analyze information.

Many e-discovery platforms and contract analysis tools have a form of conceptual search, or wish they did. Conceptual search techniques are powerful, and become even more powerful the more contracts there are. Getting the most out of your contracts, efficiently and thoroughly, is a tough process that can’t fit into just one blog post. To learn more about word embedding techniques and other types of conceptual search that ContraxSuite deploys, reach out to us at contact@lexpredict.com or in the contact form below.

 

About Elevate

Elevate is a global law company, providing consulting, technology and services to law departments and law firms. The company’s team of lawyers, engineers, consultants and business experts extend and enable the resources and capabilities of customers worldwide. In November 2018, Elevate acquired LexPredict, the legal AI technology consultancy that created ContraxSuite. More at elevateservices.com

Comments are closed, but trackbacks and pingbacks are open.