Algorithm development is traditionally approached as a technical optimization task. Yet this process often unfolds within complex sociotechnical systems of engineers, developers, domain experts, policymakers, and others. This paper develops a theoretical framework for democratizing algorithm design, and framed as a network reconfiguration problem. By integrating conjoint analysis and human-in-the-loop (HITL) methods, I propose a rethinking of how development of algorithms is approached, and specifically the mechanisms that contribute to influence within the network. Namely, new edges are created between peripheral stakeholders and central developer hubs, while iterative feedback loops re-balance flows of authority and trust. This theoretical advance in modeling sociotechnical systems as adaptive networks is premised on the notion that for algorithms in consequential settings to have a chance at deployment at scale, they must be viewed as reliable. And reliability is heavily informed by end-user attitudes toward the system. To illustrate, I present a case study in healthcare, where algorithm-aided hybrid decision-making tools assist physicians/providers in consequential settings. The case focuses on how conjoint experiments help to reveal tradeoffs in human preferences and priorities, and then how HITL allows these preferences to be embedded into the evolving system design.