NFPA Journal - Spring 2024 - 43

between when incidents happen and when they
are collected in reports.
If NERIS can deliver on its real-time-data goals
while maintaining the depth and breadth of data
that NFIRS offers, the new system would be a " total
game changer " for research-and by extension, fire
safety-Messerschmidt said. " It will make it possible
for us to do even more in-depth studies, and in
much closer to real time than what we have done
before-that will help us stay ahead of more emerging
issues, " she said. " But all of that remains to be
seen. I'm cautiously optimistic. "
'A very narrow path'
Messerschmidt's tempered optimism for NERIS is
shared among others in the research community,
who are trained not to speculate but to wait and
see where the evidence points. Based on meetings
she's attended as an advisor to the project, Messerschmidt
says she's " happy with the approach they
are taking, and I think they have a good foundation
for success. " Like others in her field, however, she
remains eager to see the types and quality of data
that the new system will produce.
Of particular concern is whether NERIS will match
the breadth of data that NFIRS has offered for decades.
To ease the load on overworked fire departments,
USFA and FSRI have opted for a more simplified incident
dataset-the data the fire departments must still
manually enter into the system-and are making tough
decisions on which NFIRS variables to eliminate from
the new system's incident module.
As a result, " we might not get as much detail as we
are used to on some things like we did with NFIRS, and
that's what I'm curious about, " said Messerschmidt,
adding that she's also sympathetic to the effort to
reduce the amount of time firefighters spend on data
entry. " There's no doubt that this system will provide
us much more solid data on the response side. But the
incident data is what we truly need to inform fire safety
messaging, codes and standards, and the rest. And
that still requires somebody to go in there, collect the
information, and enter it into the system. We haven't
yet seen what kind of data they will collect and the
kind of variables that will be included. Until we know
that, it's difficult to say how much is going to impact
our work, good or bad, going forward. "
To satisfy both objectives, the developers of NERIS
have been walking a very narrow path. " We want to
ask the minimum number of questions to get the
most accurate data-that's kind of the mantra we're
following, " Weinschenk said. " We understand that,
once we put something in, removing it down the line
becomes significantly harder. We're also very cognizant
of the value added in any additional question
that we're going to ask the fire service to complete.
It's certainly been a struggle for us. "
What compromises will be made, and how they
might impact the incident datasets that researchers
rely on, is still a big unknown. But Weinschenk
thinks it is still possible, through smart design, for
NERIS to achieve both objectives.
One way, he said, is to supplement the fire incident
module with a follow-up module that would
be completed by fire investigators in the weeks and
months after a fire. Details such as a fire's cause and
origin are notoriously difficult to know just hours
after a fire and are often tagged as " unknown " in
NFIRS and never updated. " But if that fire is going
to be investigated, we should ask that person doing
the investigation those questions, " Weinschenk said,
with the objective of creating a more complete picture
of an incident than what NFIRS can create.
Another way that NERIS designers are working
to maintain robust incident data is through a better
design for the system of questions that firefighters
need to fill out after an incident. Like NFIRS, NERIS
will rely on a kind of decision tree to capture data
from firefighters. The next question the system asks
depends entirely on the firefighter's answer to the
previous question. The NFIRS module is notoriously
easy to game-if someone wants to cut the session
short, they only need to know what answers to provide
to get to a dead end, and the questions would
stop. Answering " unknown " is usually the surest bet.
To fix this, the development team has been focused
on designing the module that asks more streamlined
questions in the hope that responders will complete
it faster and more accurately. The developers have
been guided in the effort by research on human
behavior, and even studies on NFIRS user behavior.
One such study was conducted by Marty Ahrens, the
former manager of fire analysis services at NFPA,
which looked at factors impacting data quality in the
fire service. Among the report's findings was that,
as the number of drop-down options increases, the
response and accuracy rate plummets.
Weinschenk thinks that by designing a smarter
module, with simpler questions that are faster
to answer, NERIS can find the sweet spot where
firefighters save time while also delivering more
accurate data. And once fire departments see what
NERIS can do with that data, the incentive to feed
the system more data will only grow, like a snowball
gaining mass as it rolls downhill.
" If we relieve some of the burdens of entry,
hopefully we can increase participation. And if we
provide value back to that data, we can also potentially
increase participation, " Weinschenk said. " The
more participation, the more confidence we have in
our aggregated data, the more value we can provide
back. It's this self-improving cycle-if we can get
everything else right. "
That 'if' is the big question that researchers, fire
departments, standards developers, and government
officials are eagerly waiting to answer.
JESSE ROMAN is senior editor of NFPA Journal and host of
the NFPA Podcast.
NFPA .ORG/JOURNAL * NFPA JOURNAL | 43
http://www.NFPA.ORG/JOURNAL
NFPA Journal - Spring 2024

Table of Contents for the Digital Edition of NFPA Journal - Spring 2024
