Thursday, April 12, 2018
9:30 AM - 11:30 AM
Assessment Working Group
These broad session notes attempt to capture the spirit of the discussion and should not be interpreted as a transcript. Although Ed-Fi Alliance staff were involved in capturing these observations, the notes below should not be construed as official, complete, or 100% accurate.
Notes to accompany the slides
Governance process has been working well in several areas: participatory, seeks out expertise, how to make decisions and execute upon them for addressing needs of the community.
BUT, the scale and growth of the movement and community caused the consideration to review and research governance models that will build the structure to allow Ed-Fi to grow larger more successfully while maintaining community involvement and voice balanced with fidelity and sustainability.
Ensure alignment of Ed-Fi direction with Ed-Fi community needs.
Governance is multi-tiered, and will be a 2-way process of review/clarification/approval, etc.
As the governance process evolves, what about the license? Is it a topic for governance to consider changes to licensing to move toward more open source and correlated impact/changes to IP?
Yes, the governance structure is exactly why and how this would be discussed / considered / worked through
How do we expect vendors to commit and work toward these standards/certifications before workgroups are spun up to inform those efforts?
Refer to the TAO of the Ed-Fi Alliance. We start and begin work to effect change and participation and involvement and then fine tune, iterate and improve the work in these emerging areas.
How do we protect and balance the demand side (agencies/customers) with the supply side (vendors/providers) such that as we seek sustainable directions they aren’t co-opted by powerful supply side interests?
We will prioritize raising awareness on the demand side along with commitment to participate, pull in domain expertise to flesh out use cases for agency IT groups, and provide a demand side balance with the supply side perspectives.
How do we get involved?
See Nancy Wilson to join the sign on sheet today. All attendees today will get an email next week about next steps. Can also contact anyone on the Ed-Fi team by email, etc.
Priority Use Cases and examples from attendees
Teacher developed assessments and item level data with teacher authored items, need to be able to do stats analysis, scale scores, and return that back post analysis for learning/instructional intervention
Which answer chosen, not just right/wrong
Cross reference standards with questions, like a blueprint
Type of assessment, mapped to multiple standards (CC, but also SEL, behavior frame, etc.) and mapped to multiple scores, and variable scores
Looking at multiple grade bands. A standard that spans multiple grades.
Competency frameworks. See Chris sturgis & Susan Patrick “competency works” at iNacol
More info at: https://www.competencyworks.org/
Item types; constructed response versus selected response
Learning standards, broken down to learning targets
CASE specs. How to we support or interact/intersect?
Gamification data; usage/utilization type data
Competencies vs standards
Use of student assessment data for teacher effectiveness assessment and program evaluation
This is an area to be very careful with. It’s a lightning rod issue politically that could ignite a huge pushback from unions, states, etc.
Self assessment and SEL
Performance based assessments
Question around how flexible the tech stack is to support the various types of items/data needed by these assessment considerations
Yes, flexible. Can describe these things in metadata, point to actual URI of the content (eg. like portfolio items, etc) BUT, fleshing out these requirements could really help to justify those tech stack needs.
Eric Jansson will be point of contact for this work. firstname.lastname@example.org
Open questions and areas of development for workgroup consideration
Identifiers for learning standards
Diversity of ecosystem metadata
Who owns controlled vocabularies?
One API or Two?
Relationships to “gradebook” data
General positive reaction to governance/workgroup model.
Lack of participatory model can be a block to adoption.
Concern re maintaining active agency input for specifications, both due to inequities in finances but also due to difficulty for agencies to finance participation.
Developed an initial list of use cases, and will likely need to prioritize these as there was considerable diversity.
A number of domain data model issues, including:
Representation of grade bands
Applicability to other kinds of assessments, like SEL assessments
Connecting back to item-level data and responses, including for technology-enhanced items