The wiki explanation of stress concentration factor goes as follows:
<<""Concentration factor
The maximum stress felt near a crack occurs in the area of lowest radius of curvature. In an elliptical crack of length 2a and width 2b, under an applied external stress σ, the stress at the ends of the major axes is given by:
where ρ is the radius of curvature of the crack tip. A stress concentration factor is the ratio of the highest stress (σmax) to a reference stress (σ) of the gross cross-section. As the radius of curvature approaches zero, the maximum stress approaches infinity. Note that the stress concentration factor is a function of the geometry of a crack, and not of its size. These factors can be found in typical engineering reference materials to predict the stresses that could otherwise not be analyzed using strength of materials approaches. This is not to be confused with 'Stress Intensity Factor'."">>
This kind of suggests to me that the factor you refer to is an emperical number found to be sufficient to reduce the stress concentration enough for it not to be a problem.
It also says that most factors can be found in engineering tables for those applications and materials.
As it states that the stress concentration rises to infinity as the factor approaches zero, it indicates that with a too small factor, the material will fail due to the concentration of stresses. This must be depending on application and material so i suggest you follow the wiki links and look further into the subject taking into acount your application and materials.