⏰🔍💊faster measurement merges behavior identification and algorithmic prescription

[hyunjimoon]

Ultimately, I'd like to persuade you that every intervention should be algorithmic. However, this requires understanding how useful phenomena evolves faster than research methodology and measurements. Based on ## 1 will discuss this.

My definition of "algorithmic" is "initial and termination state can be identified, has updating logic, and finishes in a finite number of iterations".

Just as chef assembles ingredients under certain recipe with the help of measurement, researcher combines phenomena under theory with the help of measurement. I established analogy as the table below. Do you have anything comments/questions/additions for the last row "how to adopt when $\theta_t$ evolves faster than $\phi_t$?", "researcher" column?

Angie's Q:

I became curious whether development of measurement technologies (along with cheaper intervention cost), would create incentive for the two phases (first testing existence of behavior then testing effectiveness of intervention) to integrate. I may be wrong, but most behavior OM research including this seems to assume, after the first identification test, everyone shows a similar degree and direction of behavior. But as your research on different degrees of risk perception in different countries show, degrees (and sign) can differ across time and space. This led me to question, assuming we can measure every state of an agent and its environment that matters to an agent (including belief and goal), do we need the concept "behavior"? Just as all unmodeled uncertainty is jumbled into process noise (if you agree), is the unmeasurable decision making mechanism of individual (agent) that can be measured in population level abstracted as "behavior" so that it can be further plugged in to individual decision model, in order to make it more realistic?

H answered:

behavior as observable actions of individuals (and organizations etc), while 'behavioral' signified to me an approach that takes the empirical observation of agent's behaviors seriously in determining the underlying mechanisms (e.g. their policy, utility function, habits etc) that creates those behaviors. Typically behavioral is referred to in comparison to a more axiomatic alternative, e.g. the rational choice theory, which starts with a more theory-driven assumption on the underlying mechanisms that create behavior (often an optimization to maximize agent utility with some constraints) and drives the resulting expected behaviors. So in short when you think about behavioral it is helpful to contrast it with the alternative (of e.g. rational choice) rather than thinking of it in abstract/stand alone.

	chef	researcher
input1 raw material	fresh ingredient e.g. vegetable, texture molecule	timely phenomena e.g. research questions from cancer, brain, ai, entrepreneurship
input2 process	past recipes	past theories
input3 measurement tool	measurement tool e.g. thermometer, scale	survey, interview, experiment, simulation
output	cuisine	paper
input123 $\rightarrow$ output	cooking which assembles ingredients under certain recipe with the help of measurement	research which combines phenomena under theory with the help of measurement
desirability of user	- eater who's goal is nourishment need higher importance weight on ingredients - gourmet would care more about recipe and measurement	- general public or lay audience may prioritize accessibility and applicability of findings - academic or specialized audience may focus on methodological rigor, theoretical contribution, and relevance to existing literature
desirability of producer	easier, cost efficient, user-satisfying, getting good review, selected as michelin star	easier, effort efficient, meaningful research, paper being cited, published in top journal
efforts to maintain process consistency	- buy ingredient by bulk and freeze (cabbage for kimchi) - test different combination of ingredient and recipe on target customer	- conduct systematic reviews and meta-analyses - perform pilot studies and pre-register designs - follow established protocols and engage in open science practices - tailor research outputs to audience needs
process step1: understand desire from sampled user	understand the taste of different target customers $\theta_t$	- learn important phenomena in the field by going to seminars and talk - attending interdisciplinary conferences to gather diverse research questions and trends
process step2: develop feasible and viable process for desired product	develop recipe $\phi_t$ for each target customers $\theta_t$	Develop a research methodology $\phi_t$ tailored to each specific research question $\theta_t$, involving a combination of qualitative and quantitative methods e.g. designing a mixed-methods study that combines surveys and interviews to explore new aspects of consumer behavior in digital marketplaces
process step3: make process scalable to maintain quality of design and compliance	Standardize cooking processes and ingredient sourcing to ensure consistency	- reproducible and replicable research - ensure the scalability of research practices by adopting standardized data collection and analysis protocols that can be applied across various studies for consistency e.g. developing a template for data management and analysis that ensures consistency in research practices
how to adopt when $\theta_t$ evolves faster than $\phi_t$?	1. modularity (modular recipes with interchangeable components that can be easily modified to create new dishes) 2. innovative culture (experiment with new techniques and ingredients) 3. flexible supply chain management (quickly adapt to new menu needs)	1. Modularity 1.1 Develop Modular Research Frameworks: research designs with interchangeable or adaptable components, such as modular questionnaires or flexible experimental setups, that can be easily adjusted to study new phenomena or incorporate emerging theories. This approach reduces the time needed to design new studies from scratch 1.2 Utilize Existing Data Sets: use existing data sets for preliminary analysis or to test new hypotheses. Need final clean test on newly collected data set though... 2. Innovation 2.1 Promote Interdisciplinary Collaboration: bring in fresh perspectives and methodologies that can be adapted to your research area 2.2 Rapid Research Prototyping: preliminary studies or exploratory analyses that can be quickly executed to test new ideas or methods. This approach supports the iterative refinement of research methodologies in response to new developments. 3. Flexible Supply Chain 3.1 Build Collaborator Network: Establish relationships with a broad network of collaborators, including other research institutions, industry partners, and cross-disciplinary teams. This network can provide rapid access to new theories, data sources, and analytical tools. 3.2 Adopt Open Science Practices: Engage in open science practices by sharing data, research instruments, and methodologies. This can facilitate the rapid dissemination and adoption of innovative research methods and encourage collaborative improvements. 3.3 Utilize Agile Research Methods: Implement agile methodologies, traditionally used in software development, to manage research projects. This involves breaking down large research questions into smaller, manageable tasks with short cycles of planning, execution, and evaluation. This allows for quick adjustments in response to new findings or changing phenomena.

(TBC)

2. two phases will start to overlap

Given the above is acceptable, next is whether two phases in the first and second column below will overlap. Currently, the first step "behavior identification" tests null hypothesis "behavior A doesn't exist". After falsifying this, researchers designs intervention prescription and further tests second null "intervention B is not effective". Representing the existence of behavior with $\theta = I(\text{measurement on population C supports existence of behavior A})$, ideal procedure of current empirical OM paper is: $\phi$, $p(\theta=1, \phi=1) = p(\theta=1|\phi=1)p(\theta = 1)$

= p(\theta|\phi)p(\phi) = inHowever, greatest leap of faith is applying

 for instance, by sensor technology that extracts implicitly, the two will begin to overlap. My question is, how is HT1 + HT2 overlapped which in some is what recommender engine is 

steps	behavior identification	intervention/algorithm prescription
null hypothesis	behavior doesn't exist	algorithm is not effective
e.g.	- competition influence the inventory holdings of General Motors’ dealerships operating in isolated U.S. markets (Olivares09) - increase in global sourcing results in an increase in inventory investment (Jain14)	Fast Learning and Pricing for Varying Assortments algorithm (Ferreira23)
increase of speed (sensor)

Comparing the intervention from our reading lists, interaction can be divided into one-time or multi-time. Having fairness and efficiency of an algorithm as two main objectives, fairness should start from identifying the existence of  Between studies that try to identify the existence of deviation (e.g. motivation among race, deviation from profit maximizing value)Comparing examples from one-time (What Motivates Innovative Entrepreneurs?) vs multi-time (Demand Learning and Pricing for Varying Assortments), one-time is more multi-time is providing new tool  intervention depending on how complex the underlying data generating process is and behavioral components and cognitive bias (e.g. perception of being in the last queue) increases complexity.

Another way to visualize complexity of intervention is depth of intervention tree. If we give homogenous treatment to every person, then tree depth is 1. However, if we customize the intervention by sequential step of his/her prior elicitation, then the tree becomes deeper. I had a feeling most people only defines the latter intervention as "algorithmic". Is this correct?

3. connecting with optimization algorithm

Viewing "cut" as hypothesis to be falsified on the existing theory and hypothesis of human behavior, we are using data to falsify.   Then by using different algorithms (benders decomposition or column and row generation in optimization algorithm ~ profit maxizing algorithm) the system is converging to (unknown) ideal formulation. This could be the form of decision tree or cut generating program.

Although Kris crossed the line between algorithm in management and optimization context, I beg to differ. Gist of integer optimization algorithms like column generation is how to systematically add cuts to the integer constraint (AX>b) relaxed polyhedron ($F= {X|AX>B, X \in R}$) to ideal formulation $conv(F \cap X \in Z)$. Changing gear from geometric to algebraic perspective, systematic cut is a constraint that can be added to different subgroups. My guess is, systemic lab experiment design that are replicable is

1. hypothesis design (math model or existing literature)
2. survey design
3. run pilots
4. pre registration
5. run full study
   kris gave tips "let the reviewers decide what they want to do (if we don't have experiment E, would this be a rejectable component?"

[hyunjimoon]

Charlie: This presentation on how a person spends his/her time — Maybe same model can be used on a different time scale for how an entrepreneur spends his/her time over the course of nailing, perhaps?

Angie: yes but definition of “behavior”, “interpretable” I think we need to be very concrete; Currently the framework is identify behavior then optimize this but as measurement increases this framework may change: #198. Separating industry effect (I_0) vs startup effect (S_0) so that I can create counterfactual of S_1 given I_0 is my approach and hardwareness is example of industry effect (Assumption: clockspeed_I << clockspeed_S)

based on Charlie's comment, I asked Yuebing whether she can share the code/logic on how you set finite number of different activities (goal is to understand behavior in startup operation)

[hyunjimoon]