I tried to re-read this now a few times. And I’m still stuck…
I do think that you don’t use the proper algorithm here. What are these both layers,?
I tell you how the algorithm works:
Both layers have to be point layers and it will calculate from each point in layer A (start) a route to each point in layer B (destination).
I still don’t know what you’re trying to do, but it doesn’t seem like what I just described.