This has actually happened to me, running touch for the local club on a Saturday "off". Pile of bodies rolled into touch, so I put up my flag and when the ref approached I had say sorry, but I have no idea who was in possession. No pause, no doubt, "right, attacking team throw in."
I was impressed at his speed of decision and I am even more impressed now after reading this thread!