Studies have shown improvements in speech understanding and quality of life with the use of hearing devices. However, even with that improvement, individuals do not always show the same performance in real-world communication settings. Considering that the communication process involves multisensory integration, this study examined speech recognition performance and brain activity in adults with normal hearing (NH) and hearing loss (HL) in auditory only (AO), visual only (VO), and audiovisual (AV) conditions. Twenty adults with NH and 20 adults with HL completed a speech recognition test while electroencephalography (EEG) was recorded in AO, VO, and AV. Power spectral density values were analyzed for the delta, theta, alpha, beta, and gamma bands. The NH group showed significantly better speech performance than the HL group in AO and AV, while the HL group showed significantly better performance in VO. Regarding EEG characteristics, significant condition effects were observed in the theta and alpha bands in the left hemisphere and in the delta, theta, and alpha bands in the right hemisphere. A significant group-by-condition interaction was observed for the gamma band in the left hemisphere, with the HL group showing greater activity in VO and AV. The findings suggest that multisensory speech processing differs according to both sensory modality and hearing status, and visual input may play an important role in individuals with HL.