Community Ideas & Best Practices | Blog | Forj

The Difference Between Knowing What Members Did and Understanding What They Need

Written by Terri Herrmann | Jul 9, 2026 5:22:21 PM

Two Associations. Same Data. Completely Different Outcomes.

Two association education directors receive the same weekly analytics report. The same course completion percentages. The same search query volume. The same evaluation scores.

Director A reviews the numbers, notes that completion is up slightly, and moves on. The curriculum planning meeting next week will look a lot like the one last quarter.

Director B looks at the same numbers and asks a different question: what are members searching for that we don't have content on? She cross-references the search terms against the course catalog, finds six high-frequency queries with no corresponding resource, and shows up to the next meeting with a list of gaps instead of a list of opinions.

Same data. Completely different outcome. The difference isn't data access. It's the question being asked.

What Reporting Looks Like vs. What Insight Looks Like

Most associations that review their data are in reporting mode. Reporting answers the question: what happened? It tells you the completion rate, the attendance number, the average evaluation score. These are useful baseline metrics. They are not, by themselves, useful for curriculum planning.

Insight answers a different question: what does this mean? It takes the raw activity data and asks what it reveals about member needs, gaps in current offerings, and trends worth addressing.

Here's what that shift looks like in practice:

  • Course completion rate is 68%Insight: Members are dropping off at Module 3 — something in that section isn't working, and that's the real problem to solve.
  • Search term "AI compliance" appeared 340 timesInsight: Members are facing an emerging regulatory challenge that your catalog doesn't address yet.
  • Webinar registration: 412 | Attendance: 180Insight: Strong topic interest, but something about format, timing, or positioning is creating drop-off before the session starts.
  • Average evaluation score: 4.2/5Insight: Open-ends reveal consistent frustration with pacing — members want more time on application, less on overview.
  • Community posts up 22% this quarter Insight: Questions about a specific topic cluster are spiking — a new member need is emerging that your content calendar hasn't addressed.

The Gut-Feel Curriculum Problem

Picture a typical curriculum planning meeting. Subject matter experts make the case for their preferred topics. A staff member mentions something she keeps hearing at the help desk. The executive director flags a topic a competitor just launched. Someone suggests doing more with AI.

The result is a content plan that reflects internal perspectives and organizational energy — not member evidence. It produces mostly fine results: programs that are competent, relevant enough, and occasionally excellent.

What it rarely produces is a catalog that feels genuinely tailored to what members need right now. Built around their actual gaps, their real questions, the competencies they're actively trying to develop.

What Members Experience on the Other Side of Gut-Feel Decisions

A member logs in to find a course recommendation for a topic she mastered three years ago. She searches for guidance on a challenge she's facing this week and gets zero results. She attends a webinar on a topic that sounded relevant and leaves 20 minutes in because it doesn't map to her actual job.

None of these experiences are catastrophic. But they compound. And when renewal time arrives, the mental accounting of "does this association actually understand what I need?" shapes the answer.

 

 

While you're building curriculum on instinct, your members are quietly updating their assessment of whether you understand them.

 

The Better Way

The shift from reporting to insight isn't complicated. It starts with a single change: bringing member behavioral data into the curriculum planning conversation as evidence, not decoration.

Not "here's how many people completed last quarter's course" — but "here's what members are searching for that we don't offer, here's where they're dropping off and why, here's what they keep asking about in community that we haven't addressed."

That shift doesn't require new technology or a data science team. It requires new habits — and a framework for knowing what to look for.

 

Want the full framework? Download the guide:

Are Your Members Shaping What You Build? A Practical Guide to Member-Driven Curriculum