This is an excerpt from my book Florida and the American Revolution

Most of the fortifications that were erected both in Pensacola and St Augustine were built largely by slave labor. While in Pensacola this was not an odd sight, in St Augustine where many free blacks had been armed (unlike on the rest of the North American continent), the use of slave labor in a traditionally intensive labor task not undertaken by local whites created a bizarre and troubling dichotomy. 

As the war wore on, some slaves ran away and joined the native Americans surrounding the town. Those caught running away were jailed for the duration of the conflict in a specially erected prison in St Augustine’s main plaza. This however still created confusion because the colony had taken in so many runaway slaves from other colonies who had become free by defecting behind British lines, and were in many cases as noted above, armed. This made it truly difficult to identify which African-Americans were in fact slaves and which were actually free. 

In reality, if an African-American asked to serve in the British forces, regular or irregular, their status was never probed and they became de facto freed as a result. While British authorities varied in their general views of whether arming blacks was a good idea or not, in time it became a general practice. Those who stayed loyal to the crown and served in government in the Florida’s, even if slaveholders themselves, largely came to view pro-independence colonists with greater contempt than any other human beings.