Brittleness is an important property of rock that predicts the rock-crushing behavior, energy consumed in rock cutting, and selection of optimal cutting geometry. The punch penetration test directly measures the brittleness and force applied to the tool during rock penetration. In this test, a steel mold and cement filler are used to simulate rock confinement. Given the flexibility of numerical methods, in this study, the effect of rock confinement on tool penetration and rock brittleness was investigated by modeling the penetration process using the finite element method and validating it with a laboratory test. The results show that by numerically modeling the punch penetration test using the finite element method, the results of this test can be obtained with high accuracy. It was also observed that the maximum force applied to the penetrator and the brittleness index in the confined state of the rock increase compared to the punch penetration test, and this increase is more observed in stiffer rocks such as granite.