Is the income tax act applicable only to residents?

It is applicable to both residents and non-residents. If you earn income in India then you would have to pay more taxes even if you are a non-resident of the country.