Yes, basically, to operate in Class B Airspace (US standards), you have to be on Instrument Rules, hence you can't have a visual approach.
I'd like to correct you regarding IFR in Class B airspace.
You are correct that an aircraft needs a "clearance" to operate in class B airspace, but that does note mean an instrument clearance. It simply means a clearance into or through the class B airspace. As a pilot you need to hear the words "N1234C is cleared into the Class Bravo Airspace"
If you listen in clear weather it is not uncommon to hear an aircraft on an instrument flight plan "cleared for the visual approach". At Class B airports or others operating within class B airspace aircraft may not be given visual approaches due to the traffic volume. By sequencing traffic onto the localizer or final approach course of an approach, the controllers have better control because those aircraft must follow prescribed instructions. Visual approaches of aircraft are common at class C and other airport when the weather is VFR.
If an aircraft on an instrument flight rules flight flies a visual approach they are still on an instrument flight plan until it is either cancelled by the pilot either in the air or on the ground at a non-towered airport or by ATC tower personnel upon touchdown of that aircraft. So, a pilot who lands at a towered airport when the tower is open never has to remember to cancel his flight plan.
You may have confused Class B with Class A. Class A is airspace from FL 180 to and including FL 600. Any aircraft operating in class A airspace, reguardless of weather conditions must operate on an instrument flight plan.