City crews dismantled a controversial homeless encampment on Vancouver's Downtown Eastside Tuesday, just days after residents vowed to fight their eviction.

Workers could be seen using a backhoe to clear tents, shopping carts and other items from the lot, which those living there named "Sugar Mountain" due to its proximity to the Rogers sugar refinery.

The remaining residents at the camp were moved into shelters Monday night, according to City of Vancouver spokesperson Jag Sandhu.

The city plans to build a 50-unit modular housing complex on the property, which will serve as temporary housing of those living on the street.

The original deadline for residents to move out was noon on Friday. As that time approached, however, emotions ran high among residents who said they had no intention of leaving.

A news conference being held by advocates for those living at the encampment ended in a skirmish involving a cyclist who alleged they were responsible for bike thefts in the area.

Video captured by CTV cameras shows a protester placing his hands near the cyclist's neck and shoving him to the ground.

Troy Tyrell, the cyclist, told CTV News he has since asked police to investigate the incident as a possible assault.

Later that day, the city said it didn't plan on seeking an injunction against those still on the property until the New Year and that it didn't expect any action to be taken until then.

On Tuesday, Sandhu said such action wasn't necessary because residents ended up agreeing to move out.

Some residents, however, complained that they weren't given enough time to gather their belongings.

"We tried asking the city just to hold off so we could get our stuff out because of the weather," said a man named Bernie. "They're saying 'No, you get what you can get and as the loader comes around they're going to clean it out.'"

Located near the intersection of Franklin Street and Vernon Drive, the Sugar Mountain encampment was set up about six months ago after officials forced residents out of another tent city on Main Street.

Modular housing is the province’s near-term solution to homelessness while it works to find longer-term fixes.

"Temporary modular housing is quicker to construct than a traditional building and will serve the immediate need to get the city's homeless residents into a safe, warm home as quickly as possible," city spokesperson Lauren Stasila told CTV News last week.

In Vancouver, there is already a modular housing project at 220 Terminal Ave., and a development permit has been approved for a lot at 650 West 57th Ave.

Some of these initiatives have also been controversial, however.

In early December, a B.C. Supreme Court judge had to order protesters to leave the lot in Marpole, which is near three public schools.

The city is seeking a permit for 4410 Kaslo St. It is also considering modular building at 501 Powell St. as part of its joint plan with the province to provide 600 temporary housing units in Vancouver.