To remove any item from the builder's scene, you have to select it and then you can either:
NB: Any removed task will automatically remove its corresponding connections.