How can I avoid it?
There are several ways you can avoid IR35 - although they may not be palatable to you, or your customers.
First, you can avoid IR35 altogether by rewarding yourself for all work done in a form subject to PAYE. IR35 was intended to stop the avoidance of tax and NICs through the use of intermediaries - if there is no avoidance, there will be no IR35 liability.
Second, you could give up your current arrangements and seek to join the staff of your main customers. This might get around IR35, but it will limit your options and increase the costs of your customers (through NIC, holiday pay, etc.)
Finally you could examine and, where appropriate, re-write your contracts. A great deal of care must be taken, not only to ensure that the terms of your contract(s) are such that IR35 will not apply, but also to ensure that they will stand up to HMRC scrutiny. In particular, HMRC might seek to prove that although your contract might on the face of it be one to which IR35 could not apply, in fact the manner in which your relationship with your client operated was one which fell within IR35. In this case HMRC will insist that IR35 be applied to earnings under the contract.
If you wish to put your contracts to the IR35 test, HMRC will give its views - but remember that this view will be based on the information provided. If, in fact, IR35 should apply, HMRC will not be bound by any previous decision.
All public bodies, including local and national government, and publicly owned companies such as the BBC and Channel 4 are responsible for deciding whether the contractors they engage are subject to IR35. If they conclude that they are, the user (or agency through which they are recruited) will be required to account for the tax and National Insurance contributions through the RTI system. If the contractor company is engaged through an agency, the agency (or the agency nearest to the contracting company if there are several) will be responsible for dealing with this decision and accounting for the tax and NIC.