Ok to close out the thread...
I pulled all the caps out and tested them, and since it wasn't actually as expensive as I thought it was going to be, I ordered replacements for all of them.

C3, C4: rated 10uF 200V; measured 10.2uF, ESR=2.8ohms; new caps: 10uF, ESR=1.0ohm
C8, C20, C21: rated 33uF 50V; measured 34uF, ESR=0.8ohms; new caps: 30.4uF ESR=0.6ohms
C9: rated 220uF 16V; measured 196uF ESR=0.9ohms; new cap: 193uF ESR=0.5ohms
C11: rated 56uF 16V; measured 51.4uF ESR=1.2ohms; new cap: 51.4uF ESR=0.47ohms
C14, C19: rated 680uF 10V; measured 787uF/675uF...

My first question would be whether those ESR values seem high for the application. These caps are 200V rated so I take it they are seeing the line voltage on the primary side. So would ESR matter that much for those caps.

Ok so I pulled the 2 200V 10uF caps, they measure OK. No sign of bulging, the legs looked fine.
One was 10.2uF ESR=2.8ohms, the other is 10.2uF ESR=3.9ohms.
These caps are marked JP, where the J and P are "combined" (the straight part of the J is the same as the straight part of the P)
other markings are CE-TKM, 105 C, 08-02

So what's the course of action, keep removing caps and testing them?

Sorry for the delay, I didn't get around to this until today... I didn't have any 47uF 50V on hand but I did have some 33uF 50V so since they are the same as the original, I thought I'd try that.
Result: no difference. I see a pattern in the behaviour when trying to power this set top box on. After about 9 or 10 times pressing the power button, the standby LED turns off for a split second, but then comes back on. (normally when this box is ON, the LED is off). So I keep pressing the button and after another 9-10 times, the LED blinks. Repeat this about 4 or 5 times in total, and then the...

Well the "it fixed itself phase" was short lived. The set top box started exhibiting the same behaviour again. I just keep pressing the power button (either on the remote or on the front of the box) repeatedly until it stays on. Takes about 20 button presses before it holds and stays on. So it seems like some poor joint somewhere that is getting intermittent contact, but I took it apart again and looked at all 3 boards on both sides, and nothing stands out, I don't see any cracked solder joint etc. But looking at all the different sizes of capacitors on all the boards, getting replacements...

I watch some basic TV using an ATSC tuner set top box which outputs HDMI to my LCD TV. The brand is Centronics, which seems like a fairly low cost (ie cheaply made) product. The UI is a bit crude looking compared to what I've seen on TVs that have the ATSC tuner built in (but it works!). Anyway, the set top box has always had an odd behaviour, which is that turning the device on using the remote would sometimes not "stick", that is, it would turn on momentarily and then immediately turn off. It didn't do this all the time, but it did this randomly since I've had it, which is 2009. Recently...

Mech eng background, and mostly have tinkered with car repairs/maintenance, but recently have ventured into fixing electronics, mostly to keep them going. One was an old, free-to-me LCD computer display. I used it for several years, and suddenly it wouldn't turn on. Replaced a bad capacitor on the power board, and it was back in business. The other was a garage door opener. Again, replaced a capacitor in the power section of the board and got the door opener working again.