In this paper, we mainly study various notions of regularity for a finite collection {C1,…,cm} of closed convex subsets of a Banach space X and their relations with other fundamental concepts. We show that a proper lower semicontinuous function f on X has a Lipschitz error bound (resp., ϒ-error bound) if and only if the pair {epi(f), X × {0}} of sets in the product space X × ℝ is linearly regular (resp., regular). Similar results for multifunctions are also established. Next, we prove that {C1,…,Cm} is linearly regular if and only if it has the strong CHIP and the collection {NC1(z),…,NCm(z)} of normal cones at z has property (G) for each z ∈ C := ∩mi=1Ci. Provided that C1 is a closed convex cone and that C2 = Y is a closed vector subspace of X, we show that {C1, Y} is linearly regular if and only if there exists α > 0 such that each positive (relative to the order induced by C1) linear functional on Y of norm one can be extended to a positive linear functional on X with norm bounded by α. Similar characterization is given in terms of normal cones.