In an RGB document, the palette just serves as swatches, it does not control anything on your canvas. Changing the palette only affects the canvas in Indexed mode.
If you want to replace a colour in RGB mode without switching to Indexed mode, you can use the Paint Bucket Tool with the “Contiguous” option turned off.
Edit: There’s also a Replace Colour option in the Edit menu for modifying colours in your canvas.
Can you clarify what you mean by that? If you want to change your swatches, then you just change them - make sure your palette is unlocked (the lock at the top should be open), click your swatch, and modify the colour as you wish.
If you want to modify the colour you just changed in an RGB document, then use the non-contiguous Paint Bucket Tool.
If you want to do both at the same time, then you need Indexed Mode, since in RGB mode, the palette and the canvas are independent.
The corner that appears on your selected colours is coloured either black or white depending on which would be more visible. Light colours get a black corner, dark colours get a white corner.