Solar modulation of galactic cosmic rays around the solar minimum in 2019–2020 looks different in the secondary neutrons and muons observed at the ground. To compare the solar modulation of primary cosmic rays in detail, we must remove the possible seasonal variations caused by the atmosphere and surrounding environment. As such surrounding environment effects, we evaluate the snow cover effect on neutron count rate and the atmospheric temperature effect on muon count rate, both simultaneously observed at Syowa Station in the Antarctic (69.01° S, 39.59° E). A machine learning technique, Echo State Network (ESN), is applied to estimate both effects hidden in the observed time series of the count rate. We show that the ESN with the input of GDAS data (temperature time series at 925, 850, 700, 600, 500, 400, 300, 250, 200, 150, 100, 70, 50, 30, and 20 hPa) at the local position can be useful for both the temperature correction for muons and snow cover correction for neutrons. The corrected muon count rate starts decreasing in late 2019, preceding the corrected neutron count rate which starts decreasing in early 2020, possibly indicating the rigidity-dependent solar modulation in the heliosphere.