Look at it this way; as a result of Thatcher putting her weight behind the Single European Act, and then as a result of Maastricht. the SM was created which gives us effectively a domestic market of 500m people. We pay into the budget for the institution - the EU - that runs that market and provides other benefits across the Union.
And if that was all the EU did, then I am sure the Brexit result would have been different. But how many people voted Brexit, to leave the political union, but wanted to remain part of the Single European Market? (so called "Soft Brexit" now).
The biggest issue with the EU was it was formed as a Common Market, and then the political side of it was expanded. (In fact when the UK voted to remain in the ECC in 1976, they were told it was only a single market, not a political union). Those seeking greater integration, moved the agenda forward. And not necessary in a sensible order.
The Euro was brought in, not at a sensible economic time, but as a concession to France to allow the reunification of Germany. There was never proper control over individual budgets of countries, there was never a set up of fiscal control, or financial transfer from rich to poor, etc. Countries lost the ability to devalue, but didn't gain other benefits, and the European South is feeling that now - Greece, Italy, Spain, Ireland....
The Schengen Area was created, allowing internal borders to be removed, but without the necessary structures in to place to strengthen the outer borders (so cost of protecting the area, fell to those at the edge).
The Lisbon Treaty (or EU Constitution as it was meant to be) was bought in, despite 2 votes against it (reran to get the "right" result), which gave the ECJ the right to overrule on any subject that was not in a written constitution.
And lets not forget that the budget has not passed an audit. The accountants have always refused to sign off the accounts. And often the Fiscal rules are broken on a daily basis.
The EU as it stands currently, is a huge Institution, that has grown organically, and pushed in directions that perhaps were not right. It needs a huge overhaul. It needs to be brought down to earth, to become more democratic, more accountable, and to be given a clear "mission statement" that it should act towards. (Is it a single market, is it a political union, is it......?)
I think most people would happily belong to a Single Market, and pay into a regulatory body that oversaw that all played by the rules, and had the power to enforce. And if that meant there was freedom of movement of labour, goods, capital and services, then I am sure this would also be acceptable to most. But that is not ALL that the EU is.
The issue with the referendum was - it was in or out. No clear policy on what Out was, and the re-negotiation that was given for the In, was effectively "no change". So there was a clear view of what In meant - to carry on as usual. Out was more of a risk, but if you think that the status-quo is wrong, then you need to try and change.