David Cameron is ready to consider a “plan B” to curb EU migration to the UK, which would involve strict new limits on benefit payments to out-of-work migrants rather than those in jobs, as he seeks to cobble together a new deal for Britain in Europe.

The prime minister will take centre stage at a working dinner with European leaders in Brussels on Thursday evening, where he will say that concerns about migration are a major issue for the British people and that they need addressing before an in/out referendum to be held by the end of 2017.

He is expected to abandon his protracted battle for treaty changes – which would ban EU migrants who are in work from receiving benefits until they have been in employment in the UK for four years – and turn his attention to other measures to reduce the “pull factors” that attract EU migrants to the UK.

Government sources confirmed Cameron’s switch to a more flexible approach, saying that “what matters most is to fix the problems, not the precise form of the arrangements”.

The prime minister’s plan to curb in-work benefits has met a wall of opposition from other EU governments, with many saying they would discriminate against workers from other member nations and cut directly across Europe’s commitment to free movement of labour.

But with other European leaders desperate for the UK not to leave the EU, and determined to help Cameron put together a package he can sell at home as substantial, the plan B option, involving curbs on out-of-work migrants, is emerging as a potential deal saver.

Successive Polish governments have said that while the four-year ban on in-work benefits is a “red line” for them, there is scope to limit benefits for those who do not have a job, such as first-time EU job-seekers, and those who have lost a job after a short period of employment.

Crucially, say the Poles and other EU governments, such changes would not require treaty change, and would not conflict with the principle of the free movement of labour as they would apply to those out of work. The changes, they say, could be agreed by the 28 member states through the faster, easier process of changing secondary EU legislation.

Government sources said that such ideas – as well as plans to allow an emergency brake on migration where it can be shown that undue pressure is being put on public services – are now in play in the talks with the UK and could be presented as significant reforms.

This week, left-leaning thinktank the Institute for Public Policy Research (IPPR) will float detailed proposals for curbing out-of-work benefits in a report to be published on Sunday before the EU summit in Brussels on Thursday and Friday.

Entitled “Unlocking the EU Free Movement Debate”, it accepts that EU migrants’ access to welfare “is one of the main issues driving public dissatisfaction with free movement” and suggests benefits could be curbed both for those who have not previously worked in the UK, and those who have lost their jobs.

The IPPR suggests one model under which anyone who loses their job before they have worked in a host country for three years would have their entitlement to housing benefit, jobseeker’s allowance or universal credit cut from the current six months to three months. Under tougher options, the safety net could be removed altogether.

As part of a deal on benefits, Cameron is hoping for an agreement that would strip EU migrants of the ability to claim child benefit if their children do not live in the country.

After Thursday’s dinner, EU leaders will focus on measures to tackle the terrorist threat, following the Paris attacks in which 130 people died, and Cameron will use the summit to call for an EU-wide ban on high-powered, semi-automatic weapons as part of a joint European effort to prevent terrorists getting hold of deadly arms. Other measures to be discussed will be the need for greater sharing of ballistics data, and improvements to data on firearms in the western Balkans.

British officials stress that Cameron is no longer bent on achieving all of his demands but is determined to get substantial wins within each of the four main areas in which he is pushing for reform. The other three demands – on which agreement is likely – are to exempt the UK from the EU’s commitment to “ever-closer union”, achieve greater protections for non-eurozone countries, such as the UK, to ensure they cannot be outvoted by euro area nations, and give national parliaments greater powers to club together to block EU legislation.

Downing Street sources said that those campaigning for the UK to leave the EU are promoting a “ridiculous caricature” of the renegotiation process by saying that the only area of importance is the four-year benefit ban, and that the rest does not matter. They said hard-line sceptics had always demanded progress on the other three areas, but now claimed they were unimportant because they did not want the prime minister to be seen to have achieved a better deal for the UK inside the EU.

On the welfare issue, government sources did not mention the four-year ban on in-work benefits but said the prime minister would use the summit “to unlock the political will necessary to find a solution, effectively giving the green light to officials to work up a solution that would both deliver on [his] objective of better controlling migration from the EU, while also being acceptable to all.

“This would pave the way for an intense period of technical work early next year with the aim of sealing a package of reforms in all four areas at the February European council.”