Fears over future of Sports Centre