People new to RBQM often make a mistake I’ve nicknamed “KPI myopia”. They focus in on one particular KPI when doing data reviews, and that leads to bad decision making. That’s because if you only look at one metric it can be misleading. You need to look at combinations of data to get the true picture. For example, if you only look at one metric such as “missing data” from a particular site, everything could look fine. There is no missing data so everything’s ok, right? Not so fast. That metric maybe fine, and indeed the site may be fine, but you also need to look at other metrics related to that site to make a properly informed decision. Metrics such as changes in data entry times, increasing numbers of queries, early terminations and changing AE rates. IF AE rates are increasing, is that because the site isn’t screening properly? Are they enrolling patients right on the edge of eligibility? You need to interrogate the data and ask searching questions to correctly identify the underlying problems that need to be addressed. The significant problems that if not addressed could result in all the data from that site could becoming invalid. Being able to correctly assess those underlying problems comes from looking at the whole picture. If you don’t do that, and focus on only one KPI, it is easy to mis-diagnose and waste valuable time and resources on a problem that is only a symptom and not a root cause.
The other issue related to “KPI myopia” is delayed action. Lots of organisations use some form of data mining tool or statistical package. But they only look at the data once a quarter, when “there’s something to look at”. If you look at the data once a month and focus on a smaller group of ‘poorly performing’ sites, you can make a rapid, noticeable and quantifiable difference on data quality. For example, getting into the detail of what’s really going on in each site, such as which ones are improving and which have not enrolled any patients, enables you to really focus your resources. The ability to intervene early to improve data quality has never been more important because of industry challenges like patient recruitment and speed to market.