Length-biased sampling and 5-partnership truncation bias in formation target stats

[smjenness]

Context

Identified during PR #71 review while implementing duration.method = \"weibull_strat\". The issue surfaced in the duration-fitting discussion but is broader than duration estimation — it affects the formation target stats (edges, nodefactor_*, nodematch_*, concurrent, absdiff_*) that come out of build_netparams() / build_netstats(). Opening as a separate issue per PI request.

The problem

ARTnet is a cross-sectional web-based survey with two non-standard sampling features that neither the legacy univariate fits nor the joint g-computation infrastructure (#61–#63) currently correct for:

1. Length-biased sampling

The survey asks about partners that were active within the past 12 months. For ongoing partnerships, the probability of being in the sample is proportional to partnership duration (longer partnerships intersect the recall window more often). For completed partnerships, a different bias applies: only those that ended within the past 12 months are included, which selectively drops older completed partnerships.

This shows up downstream:

• Respondents with longer main partnerships are overrepresented in the "has a main partner" sample, which biases md.main upward.
• Mean degree calculations conditional on age/race carry the same bias, potentially heterogeneously.

2. 5-most-recent partner truncation

Respondents are asked about up to 5 most-recent partners per layer. For respondents with more than 5, the excess partners are not observed. Two effects:

• Right truncation of partnership count: true degree is underestimated for high-activity respondents.
• Selection bias on retained partners: the 5 most-recent tend to skew longer (if "most recent" orders by recency of activity rather than start date), further interacting with the length-bias issue above.

Scope: which target stats are affected

Target stat	Affected by length-bias	Affected by 5-truncation
md.main / md.casl	yes (upward)	yes (downward cap on high-degree)
nf.<attr> (all)	yes (heterogeneous by attr)	yes
concurrent	yes	yes (especially)
nf.deg.{main,casl,tot}	yes	yes
nm.<attr> (all)	partial (mixing by duration-correlated attrs)	partial
absdiff_*	partial	partial
durations (durs.*.byage)	yes (handled in #63 phase 3 via length-biased Weibull)	tangential

Proposed approach (open for discussion)

1. Survey the methodology literature — the ARTnet Weiss et al. 2020 paper should note how the univariate approach was designed to be robust to these biases (if at all). Similar literature on egocentric network sampling corrections (Vardi 1989, Asgharian et al. 2002, Krivitsky & Morris 2017 on egocentric inference).
2. Length-biased correction for Poisson/binomial fits — the joint Poisson fits in netparams$<layer>$joint_model are currently standard MLE. A length-biased version would weight observations by 1 / P(obs | duration). Needs partnership-duration covariates on the RHS, which may not be directly available for the ego-level fits.
3. Truncation correction — fit a truncated Poisson or zero-inflated alternative for count-of-partners models. Would need the truncation boundary (5) explicitly in the likelihood.
4. Reweighting — alternative: reweight observations by inverse probability of inclusion, if we can estimate that probability.

Priority / impact

Unknown until we compare ARTnet-derived estimates against a length-bias-correct baseline. Magnitudes:

• On duration estimation (already addressed in [Phase 1.3] Joint modeling for nodematch + absdiff + dissolution (dyad-level target stats) #63 phase 3): under naive Weibull the bias was catastrophic (mean.dur estimates off by ~1000x in heavily-censored strata).
• On formation stats: likely more modest but still material, especially for attributes correlated with partnership duration.

Related

• Blocked by: completion of [Phase 1.3] Joint modeling for nodematch + absdiff + dissolution (dyad-level target stats) #63 (establishes the joint-fit infrastructure that sampling corrections would modify).
• Informs: any model projection onto a target population whose joint attribute distribution differs from ARTnet's (NHBS MSM, AMIS 2022-24 projection) — the sampling corrections would matter more in those settings.
• Discussion: see PR Duration methods: empirical + joint_lm (#63 phase 3) #71 thread, comment chain starting "why the weibull model failed".

[sgoodreau]

Hi @smjenness , lots going on in here. I'm happy to try to be of help in this, if you'd like it.

On my first read through, I wonder two things:

• Does tergm now have the ability to handle dissolution models that are duration-dependent? I thought not, and that adding this would be very difficult. Whether those are there and/or can realistically be added obviously changes the set of possible paths to pursue here.
• Let's assume for a moment that we live in a universe where one can't feasibly fit the entire distribution of relationships, and we are willing to just match one summary statistic of that distribution. How important is it to choose duration as the summary statistic to match, as opposed to mean age of extant ties? I.e. not thinking about the latter as the method of getting to the former, but as the end in itself of being the summary statistic that one matches. This also changes how we think about the existing method. I realize neither of these options is as good as actually capturing the full distribution, or perhaps two summary statstics on it instead of one (via a 2-parameter model like the Weibull). I just don't know if we have that ability (see Q1).

Keep in mind my thinking in this realm has always been informed by the fact that I took a Stochastic Processes course in grad school -- ie Markov Chains and the like -- but not ever a Survival Analysis course, for better or worse.

[smjenness]

Hi @sgoodreau — thanks for jumping in, this was really helpful framing.

Broader context for the refactor: it started when I was working on a sibling project (ARTnetPredict) aimed at using ML-based prediction to "update" the ARTnet target stats to reflect the newly available 2022-2024 AMIS survey rounds. While building that forward projection, I realized the original within-ARTnet analytic methods — univariate marginal fits for each target statistic, one variable at a time — carry conditional-distribution bias that transports poorly to populations differing from the ARTnet sample (NHBS MSM demographics, our 2022-24 projection, etc.). So before projecting ARTnet → 2024 via ML, I wanted to come back and do the within-ARTnet baseline "right": joint Poisson/binomial GLMs fit across all attributes simultaneously, then g-computation against the target population instead of applying univariate marginals. That's the through-line for #61 through #69.

On durations specifically — you're right on both counts. Because tergm doesn't natively support duration-dependent dissolution, we're stuck with geometric dissolution in the simulation, and mean age of extant ties is the right observable to target, not mean full partnership duration. The original empirical method has effectively been targeting that all along; under exponential the two coincide so nothing was lost by the historical presentation. I had implemented a length-bias-corrected Weibull mostly to see how the comparison would look and to pull out the shape parameter as a structural diagnostic. On further reflection, you're right: even a flexible survival-based approach (a Weibull adjusted for left truncation) isn't the right inferential target for our EpiModel workflow, exactly for the reason you noted: we're still simulating dissolution under constant hazard inside the TERGMs, so the mean-duration estimate wouldn't be faithfully reproduced by the sim even if we recovered it cleanly from the data. So I dropped it and retained the two methods that actually target mean age of extant ties: empirical (marginal, legacy behavior) and joint_lm (multivariate-adjusted, same estimand, more sophisticated path).

More notes on the broader refactor and decisions over on #63.

When I'm done with this ARTnet reanalysis pass I'd love to send you a draft write-up for thoughts.