Discussion about this post

User's avatar
Randy Au's avatar

Ahaha, a few years ago I was a minor stake owner of an ice cream shop and the people actually doing the hard work asked me to help them w/ my "data skillz". But since the other owners were ex-accountants, they could manage their books and costs already and there was honestly nothing data-wise worth doing.

A lot of my success as an analyst probably comes from my social science background because out of my colleagues, I'm often the first to roll up my sleeves and sit down to hand code 200+ open-ended text feedback messages in a single afternoon until my brain rots. And it's the bridging of the qual+quant that people list and stuff works out. Except a lot of people don't like the tedious work involved =\

Expand full comment
Joe Gorse's avatar

For a moment I thought you might touch on the mystical and necessary element of belief to direct inquiry.

On a jog the metaphor of water came to me: in the Oceans of Data, there is a Sea of Information that contains Observable Truth. We can observe by putting out rain buckets (experiments) to collect water for our Data Lake. The circumstances for collecting data is an exercise in belief. In science it is the belief in a model from which we generate the hypotheses. That is an exercise in judgement. The bronze water we put into the Data Lake has contaminant artifacts of the experiment including elements of bias in the observers willingness to perceive.

In the context of your story, the unstructured video data is the qualitative data which leads us closer to the Truth or Sea of Information. From which we can also glean quantitative data once we have some basis for a model.

And then there are the red herrings such as Survivorship Bias (https://en.wikipedia.org/wiki/Survivorship_bias). In the patching and reinforcing of the bullet holes from planes that returned (rather than when they didn't return), perhaps interviewing the target demographics who aren't there would have more useful especially if the pool of people interviewed was small or not representative. This is baked into our belief about what can be True and our ability to observe

Others have said this more completely and eloquently. Thank you for the article.

Cheers,

Joe

Expand full comment
63 more comments...

No posts