The key industry has replaced worn-out equipment to face global competition after the 30-year quotas ended in January 2005. The quotas ensured that developing countries had access to the key European Union and US markets.
But while their $5.5-billion investment over the past five years has given good returns, they say that cost overruns at home threaten their hopes for the future.
The issue is enormous for debt-burdened and poverty-stricken Pakistan, which relies on textiles for about 67 percent of its total $13-billion exports or over 11 percent of the Gross Domestic Product.
"A number of bed wear and home textiles manufacturers have been forced to relocate their factories to Bangladesh and Sri Lanka," said Bilal Mullah, chairman of Pakistan Readymade Garment Manufacturers Association.
Hundreds of textile companies upgraded their equipment and operations to compete with giant rivals like China and India under the quota-less regime, but high costs - coupled with regulatory and bureaucratic hurdles - have rendered them uncompetitive, he said.
"I am seriously looking at relocating my textile units to Bangladesh as the cost of manufacturing here is going beyond our control," said Shabbir Ahmed, one of Pakistan's largest bed wear exporters, who spent $25 million on modernizing his factories.
Pakistani exports to the EU dropped 33 percent in January-May 2005 compared to the same period last year, whereas Chinese exports climbed 57 percent and India's increased 28 percent, he said.
Ahmed said that he led a delegation of the Pakistan Bed wear Exporters Association on a fact-finding mission to Bangladesh in December and was now conducting "feasibility studies" on a possible move there.
The government is skeptical of the business's claims.
"Pakistan's textile industry has shown more than 17 percent growth in exports during the first 10 months of the quota-free environment," textiles minister Mushtaq Ali Cheema said.
"Of this, garment exports have increased by 48 percent, bed wear 14 percent and other made ups by 78 percent," he said referring to the trade statistics for January-November 2005.
But the industry remains unconvinced. It says that Bangladesh and Sri Lanka offer lucrative incentives to counter the cost of doing business in Pakistan, while skilled labor is also cheaper in these countries.
"Bangladesh also enjoys the advantage of women working shoulder to shoulder with men, whereas in Pakistan women are restricted from working outside except some major cities like Karachi and Lahore," said Mullah of the Pakistan Readymade Garment Manufacturers Association.
The businessmen are also annoyed by Pakistan's many regulatory agencies, accusing them of exploiting their role instead of improving working conditions.
"There are over two dozen government agencies we have to deal with and they are appeased only with bribes, a hidden price, which adds up to the cost of manufacturing here," said Mullah.
Problems in accessing bigger markets like the EU and United States - which cumulatively import over 56 percent of Pakistani textile products - have also caused problems.
The EU recently reduced anti-dumping duty on Pakistani bed linen from about 13 percent to 7.5 percent but will not give it generalized system of preference (GSP) plus status, a special rebate that it offers to vulnerable countries.
However, Bangladesh, which was part of Pakistan until 1971, has GSP status.
Minister Cheema said that Islamabad was striving for a long term textile policy to "gain wider access to the international markets". Plans to build "textile cities" in urban hubs like Karachi and Lahore are also in hand, he added.
But the industry says that the government must act quickly.
"If the problems faced by garment exporters are not addressed urgently, they will be compelled to move their factories elsewhere," said Mullah.