The secretary of state lacks clout in raising Brexit issues with colleagues in the British government, a report said.
Julian Smith does not sit on a key committee advising the prime minister about EU exit strategy.
The Northern Ireland perspective was often missing in Whitehall without a devolved administration at Stormont presenting a common view, researchers said.
The review said: "The Northern Ireland Office (NIO) lacks the clout within government to ensure that Northern Ireland issues were given the priority they perhaps deserved.
"Officials told us that the culture of 'no bad news' among ministers translated into a reluctance to address the difficult issues raised by the Irish border and need for cross-border co-operation - for example, the operation of the single electricity market."
The Institute for Government report was entitled: Governing Without Ministers - and stretched to 68 pages.
Northern Ireland has been without devolved ministers for more than two and a half years and the authors identified areas like health reform where it has had an effect.
Senior Belfast civil servants have taken part in various meetings with UK ministers.
The confidence and supply agreement with the Conservative Party has given the DUP's anti-Brexit perspective additional influence, with its MPs' votes necessary to pass any future deal, the report said.
It added: "Indeed, this has led many in Westminster to confuse the DUP line with wider opinion in Northern Ireland, something that many of the people we spoke to were eager to make clear was not the case.
"The common Northern Ireland position, which the executive would have been obliged to articulate and then feed into UK debates, and which could have influenced the course of Brexit, has been missing.
"This absence of political representation was made even more problematic by the fact that the Northern Ireland perspective has often been missing in Whitehall, too."
It said the Northern Ireland secretary was not regarded as a key Cabinet post, and prior knowledge of Northern Ireland was not a prerequisite, meaning learning curves can be long.
Ministerial turnover was also high, the report said.
Since 2010, there have been five secretaries of state and six ministers of state for Northern Ireland.
It said the NIO has reduced in stature since the 1980s and 1990s.
"During the Troubles, Northern Ireland was a key prime ministerial priority: both John Major and Tony Blair devoted significant amounts of prime ministerial capital to the peace process and supporting the process of re-establishing Northern Ireland institutions.
"It has been much less of a priority for their successors.
"As a result, the department has struggled to attract talented, ambitious civil servants, and generally has had much less clout in Whitehall.
"Consequently, Northern Ireland issues get far less attention in UK central government than it used to."
