When data is scarce, causal discovery (CD) algorithms often infer unreliable causal relations that may contradict expert knowledge. This issue is especially problematic in the presence of latent confounders due to the exponential growth in the quantity of candidate graphs relatively to the causually sufficient case. Furthermore, the lack of uncertainty quantification in most CD methods hinders users from diagnosing and refining results. To deal with these matters, we propose Ancestral GFlowNets (AGFNs). AGFN samples ancestral graphs (AGs) proportionally to a score-based belief distribution, enabling users to assess and propagate the uncertainty while accounting for latent confounders. We leverage AGFN to propose an elicitation framework for human-driven inference refinements. This framework comprises an optimal experimental design to probe the expert and a scheme to incorporate the obtained feedback into AGFN via importance sampling. Importantly, our method is applicable even in latently confounded data --- which has been relatively underexplored by the literature --- and can be easily adapted to handle different data types by choosing an appropriate score function. Experiments with observational data show that our method accurately samples from AG distributions and that we can greatly improve inference quality with (simulated) human aid.