Magnetic fields occupying the voids of the large scale structure may be a relic from the Early Universe originating from either Inflation or from cosmological phase transitions. We explore the possibility of identifying the inflationary origin of the void magnetic fields and measuring its parameters with gamma-ray astronomy methods. The large correlation length inflationary field is expected to impose a characteristic asymmetry of extended gamma-ray emission that is correlated between different sources on the sky. We show that a set of nearby blazars for which the extended emission is observable in the 0.1-1 TeV band with CTA can be used for the test of inflationary origin of the void magnetic fields.