Something’s Gotta Give, and it Will Be Consumer Faith in Science.

Testing Issues In Oregon: A Brief History of How We Got Here

Testing issues have been endemic in Oregon. They were notably present since at least 2015, which was the year, the Oregonian did a massive multi-part story covering shortfalls meant to inform residents of issues within the newly forming recreational cannabis industry. Not only were there issues in pesticides getting through the testing process, but there were discrepancies in edibles results as well. In reaction to this piece, in 2016 Oregon adopted some of the most stringent testing policies in the country. However, in 2017 the Oregonian once again did independent testing and found the system to be lacking. Also, they found that the state was not performing any sort of randomized testing on their own.

The years since have not helped labs or the state resolve these issues. In January of 2019, the Oregon Secretary of State released a document entitled “Oregon’s Framework for Regulating Marijuana Should be Strengthened to Better Mitigate Diversion Risk and Improve Laboratory Testing.” In this scathing audit, 15 recommendations are made, intending to help Oregon make up lost time in their laboratory testing standards. Reading this audit made clear that every portion of the lab and testing process had issues. Paper-bound processes constrain how labs are accredited to run tests. Unaccredited labs can subcontract tests, resulting in overtaxed accredited labs. And sampling methods were (and still are) highly suspect.

One of the shocking deficiencies listed by the report was the lack of a reference lab. A reference lab is a third-party lab that you use to help determine the accuracy of the results of the other labs. Without a reference lab, there’s no means of calibration that the laboratories can use to ensure that their results are correct. It also found that the accreditation body for the labs was inadequately staffed and was too limited in authority to ensure they could operate effectively, in addition to having many of their processes on paper. Regulators are happy to blame their woes on the lack of this lab, but there has been nothing done to address the issue since the 2019 audit.

The audit also recommends that tracking be added to METRC, the seed-to-sale software used in Oregon, to allow tracking of when testing is subcontracted. Currently, in addition to ‘lab shopping’ by producers, where they will take their product from lab to lab until they get the THC value they want, there is also a loophole in how the labs themselves subcontract those results. If a lab isn’t accredited to run a particular test, it can subcontract that result to a lab that can — but there is no indication that such a thing happened, nor does METRC have the facility to record such information.

With all of these problems around rudimentary aspects of testing, such as pesticides, heavy metals, and THC — it’s no surprise that there are some nuances in terpene testing as well. If you look at the historical results of a particular strain at a particular farm over time, you can definitely see that testing capabilities have increased, which will bring the overall terpene counts and percentages to appear higher — when it’s our testing capability that has improved.

What 250,000 THC Potency Tests Say About Oregon

I performed a public records request and did an analysis of the 250,000 records for THC tests that the state of Oregon keeps in its Cannabis Tracking System. I quickly received four years of de-identified data in quick time, and I’m grateful to TJ Sheehy and his team who made that happen.

The records request showed not only issues with the labs, but also issues with data collection and governance ( those are always to be expected in data sets like this). But even though it was a perfectly formatted imperfect data file, it still had a lot of relevant information and told quite a story.

In prior stories, I wrote about research into issues with labs in Washington and Nevada by Michael Zoorob. That research gave evidence that there were issues in the labs by examining a frequency analysis and noting the ‘bump’ that occurs around 20% THC, which is a significant number for marketing.

Here are the graphs from the Washington and Nevada Study.

Image Source: https://jcannabisresearch.biomedcentral.com/articles/10.1186/s42238-021-00064-2

Using my 20 year background in health care data analytics ( I’m even published!), I performed the same graphing on Oregon’s test results. Unlike other areas — I have expertise in this one. In this case, Oregon’s data shows the same deformation of the curve — and it’s gotten WORSE since the audit in 2019. If the state did anything meaningful to address the issue, it’s certain it hasn’t worked.

In 2019, the plateau before 20% THC for flower is very slight.

Frequency Distribution of THC test performed on Flower and Shake in Oregon in 2021. Image Source: Author

Another view with tighter grouping.. Image Source: Author.

In 2020 you can start to see the break before 20% steepen.

Frequency Distribution of THC test performed on Flower and Shake in Oregon in 2020 Image Source: Author

Frequency Distribution of THC test performed on Flower and Shake in Oregon in 2020 with tighter grouping. Image Source: Author

In 2021 it gets out of hand.

Frequency Distribution of THC test performed on Flower and Shake in Oregon in 2021 with tighter grouping. Image Source: Author

I heard how many thought that perhaps cannabis results wouldn’t conform to a normal curve — and yet — it’s obvious when digging into the data it’s been driven by a few. In fact, one lab in particular had at over 3% of their results over 33%.

Note the curve doesn’t peak at 20, nor should it — the average THC concentration in Oregon is actually at 23.3 for the year of 2021. Which is exactly where this data set peaks. I am working with Michael Zoorob to confirm these findings.

There’s one more graph that is pretty compelling. If I look at just any results that are over a ‘biologically infeasible’ result of above 36%. I’ve color-coded it by laboratory.

You can see other work I’ve done with the data on a public Tableau I’ve shared here.

8 Labs, 4 Samples: The Round Robin: The Setup

The ‘round robin test’ was my first effort at trying to understand what was going on. I wanted to send some samples to the labs to see what they reported. Tests like this are performed and are called proficiency tests, but several labs confided issues they had with how some of the current proficiency tests are run (such as not using cannabis, but a THC-spiked hemp sample).

I used the public-facing information on OLCC’s website about contact information for all of the accredited labs. I then started calling and emailing the numbers associated with their registration. Initially, 20 labs agreed to test, and I was over the moon.

Then, of course — reality set in the form of huge budgetary constraints. I talked with my state representative, who agreed that due to what I was doing, crowdfunding, etc. was off the table and he said the only legal way to approach funding was to seek it from the industry itself, unless I could self-fund. I don’t make a lot of money, and the project was going to cost thousands in various expenses — self-funding was out.

That’s when a lab stepped in to help, and another offered as well. Most labs agreed to waive their testing costs, but it would cost the experiment its ‘blind’ nature (when dropping off the samples, the labs now knew it was related to my project).

Once all of that happened, and I re-contacted labs to gauge their interest, given how everything had changed. A lot of them dropped out or ghosted me. There were even some that received a sample and didn’t participate. Crazy, huh?

In the end, I have results from 8 performed on the same 4 samples of material. And I’d say that the data is pretty interesting.

8 Labs, 4 Samples

Sample A, B, and C were all flower. Sample A is some of the craziest flower I’ve ever EVER experienced, and I’ve actually written about it before. I even had a lab ask if it was a kiefed batch: that’s how nuts flower A is.

Sample D was a distillate.

For all of these samples, I had them homogenized at ChemHistory. Originally I was going to try and build my own homogenizer using some instructions from one of the labs that said they were going to participate — but once they dropped out, I felt free to just ask for some help on this one!

Homogenization is a very important process for cannabis testing. It’s how a lab ensures that any sample is representative. This is pretty complicated for a plant like cannabis, since potency can be higher in one portion of the plant vs. another. Essentially, everything is ground into tiny bits, and those particles mixed up so much that it can be trusted to be representative of the entirety.

One of our homogenized samples looked like this:

Homogenized Cannabis. Image Source: Author.

Results

There are two sample 2’s because I typo’d the de-identification. Image Source; Author.

Why didn’t I include THC? Because Total THC is a calculated value, and can be derived from Delta-9 + (THCA*.877).

Summary Information:

You’ll notice that in the original article where I reported this, I don’t make any conclusions or real observations about the data I’m reporting. But I’m going to say something now. For something that is such a ‘science’ there’s a lot of variability in these results. Particularly troubling is Sample C. While I’ve heard that the state has sent run tests similar to this, the fact is, this sort of one-off testing won’t be enough.

How My Experiment Was Flawed

Unfortunately, I wasn’t able to ‘blind’ the labs to the fact I was the one submitting cannabis for testing. First, I had to ask them for help to defray the costs. I heard that the state did similar proficiency tests recently, but the mere fact that I heard it is troublesome — the labs were easily able to identify that the samples were being run for OLCC.

How Do Labs Cheat?

1. Using Moisture % as a way to alter the number.

Moisture percentage can impact the THC potency of a sample, and moisture content or moisture percentage is one of the ways that labs are accused of cheating. The impact of this one is going to be pretty slight, but it could be meaningful enough to make the cannabis more viable in the current marketplace. On its own, it’s not going to result in something like the inconsistencies that are seen in Oregon.

2. Using a different calculation for loss on drying.

Using a different calculation for loss on drying.

This is another trick of math that might bump a number a little bit, but not by much. There are two calculations for loss on drying:

(moisture loss) / (wet or starting weight)

(moisture loss) / (dry or ending weight)

These two calculations for loss on drying render slightly different results, and might be a way that labs change the numbers. Again, this will most often be a very subtle change.

3. Spike the sample.

There are a lot of ways to spike a sample of cannabis. This behavior ranges from kiefing the batch (which is legal in the state of Oregon for some inscrutable reason) to straight up spiking the sample with distillate or other methods. Each version of spiking a sample comes with a pretty good way of detecting it. For instance, using distillate used to be a popular way of increasing the potency of flower until it became easy to detect by looking at the ratio of delta-9 THC to THCA. The use of THCA isolates is also detectable.

Spiking samples to get higher potency values is a common and fast way to increase potency for selected results, especially if effective lab audits at the state level are not in place.

4. Sampling Cheats

While we’re talking about samples, let’s talk about sampling. Appropriate sampling procedures are supposed to ensure that the sample taken from a cannabis batch is representative of that batch. That doesn’t mean that appropriate sampling procedures are always followed. I’ve heard a variety of stories about this. I’ve heard several versions where a farmer handing a bribe and pre-prepared sample to lab samplers, and I’ve heard of farmers leaving the samplers with some kief and some time.

While Oregon has recently updated its rules to include more specific information on sampling, it also increased the batch size to 50 pounds from 15.

5. Swap Samples With Known High Tester

If you’re going to be nefarious, you might as well go all the way and replace your sampled cannabis with a known high-tester. With some of the crazy machinations I’ve heard about labs doing, this method is both cheap and effective. Highly probable that this isn’t the method used to create the sort of mess we see in Oregon, but still effective enough to cheat.

6. Using a Standard Known To Give High Results.

A ‘cannabis testing standard’ is the material that is used to calibrate testing equipment. Many of the least dilute (therefore more effective) standards require DEA licensure (cannabis testing labs are federally illegal laboratories and not eligible for DEA licensure). The more dilute a standard, the more it can impact the results and can give much, much higher potency results. Some standards are known for yielding higher potency results, and some standard companies have failed to respond when clients have raised issues with standard accuracy. ORELAP gives no direction in evaluation of standard for use in testing, nor a list of approved manufacturers.

Unlike some of the other methods, using a standard would impact every test performed on that piece of equipment.

7. Calibration ‘Tricks’

Calibration is what it sounds like — calibrating the lab equipment to accurately reflect found amounts of THC. An intentional miscalibration of the machine would skew all of the results of the machine.

Most calibration ‘tricks’ would be caught during a thorough audit of the lab’s methods. There is no way, however, to address all of the possible tricks that could be used without a comprehensive audit of the entire preparation of the standard for calibration (there’s those standard issues we talked about again).

This is another method that would elevate the results for the laboratory as a whole.

8. Incorrectly Entering Sample Weight for Calculations

When incorrectly entered with the potency information, the sample weight is enough to wildly impact cannabis potency testing results. While the equipment used to record the mass must be calibrated daily, there is nothing to prevent someone recording a lower overall mass (which would push the potency up), and very few labs have the sort of equipment that interface this information with their lab information system, much less regulatory systems. This is an ‘easy’ cheat, as recording a mass around 10% lower than actual will yield roughly 10% inflation for potency.

9. The State Doesn’t Stop Them.

My original article about how labs cheat stopped at eight ways, but here’s the ninth.

10. Dry Labbing

In this instance, the lab doesn’t even bother to run the tests, and instead just puts in numbers to satisfy their clients’ needs.

Which Method Is In Use in Oregon?

Several lab directors were willing to tell me off the record that most likely, the sort of numbers we are seeing in Oregon indicate a combination of using 6, 7, 8, and 9. It is clear that Oregon’s cannabis lab market right now is a mess. Labs are closing due to the economic impact of the lack of oversight.

They Don’t Just Lie About Potency

What I’ve been saying for the last several months is just how if a lab will lie about potency, they’ll lie about pesticides. And it ends up, I’m right.

In other words, the anti-science behavior of these labs could start costing people their health and lives.

In a stipulated settlement recently released, a lab that railed to me about other cheaters, joined the coalition! And here is the stipulated settlement against them.

The synopsis states: “SYNOPSIS: — A laboratory licensee reported marijuana test samples as “passed” during a period when their equipment was not detecting certain pesticide analytes. Because ORLAP is the agency primarily responsible for overseeing laboratory certification and they elected to recertify this lab even after this issue came to light, OLCC Executive team approved this settlement reduction to a civil penalty or suspension in an effort to act in coordination with our ORLAP partners.”

This made me laugh, but not in a ha-ha funny kind of way but that horrible empty laugh of someone who is watching something they love be destroyed. Not only does OLCC throw ORELAP under the bus for the lighter penalty, but also spells the agency’s name wrong (at least according to ORELAP’s website). With precision like that, is it a surprise I don’t totally trust them?

But it’s digging for more detail that breaks my heart even more. The settlement states:

“OAR 845–025–8540(2)(a)(B), (2)(d) — From about
October 31, 2020 to about February 11, 2021, Licensee
and/or Licensee’s employees, agents, or representatives
misrepresented the testing results of marijuana items to
consumers, licensees, and/or the public when they
discovered in September or October 2020 that quality
control measures indicated that their equipment was
unable to test for several types of pesticides, but
Licensee and/or Licensee’s employees, agents, or
representatives nevertheless issued Certificates of
Analysis stating that the samples did not contain
actionable levels of any of the pesticides being tested for
and therefore that the samples passed pesticide testing,
when in fact several of the pesticides would not have
been able to have been detected due to the equipment
failure.”

In other words, the lab knew from October 2020 until February 2021 that their equipment was INCAPABLE of testing for pesticides, and just passed those tests anyway. And, the state of Oregon did nothing about that until 2022, when they decided that a 32 day suspension would be an appropriate recourse for intentionally endangering consumers.

Something’s Gotta Give

There are going to be lawsuits. There are going to be more honest labs that are run out of business. Dishonest labs will continue to use every method in the book to cheat the numbers as much as they like — including hijacking efforts to stop them. When I started all of this work, I mentioned several times I had been threatened by a lab director when I started poking around. This will escalate, because cannabis is a huge, burgeoning industry, and testing is an economic gatekeeper to riches.

As usual, it will be consumers paying the price with their health in addition to their hard-earned cash.