We develop a general framework for the quantization of bosonic and fermionic field theories on affine bundles over arbitrary globally hyperbolic spacetimes. All concepts and results are formulated using the language of category theory, which allows us to prove that these models satisfy the principle of general local covariance. Our analysis is a preparatory step towards a full-fledged quantization scheme for the Maxwell field, which emphasises the affine bundle structure of the bundle of principal U(1)-connections. As a by-product, our construction provides a new class of exactly tractable locally covariant quantum field theories, which are a mild generalization of the linear ones. We also show the existence of a functorial assignment of linear quantum field theories to affine ones. The identification of suitable algebra homomorphisms enables us to induce whole families of physical states (satisfying the microlocal spectrum condition) for affine quantum field theories by pulling back quasi-free Hadamard states of the underlying linear theories.