The littlest of rascals was born last week to Rascal Flatts’ Jay DeMarcus and his wife Allison, but only now is it “finally sinking in” for Jay.

In a new interview with People magazine, Jay reveals he performed two more shows this weekend before he returned back to Nashville.

“I literally got back [Monday] morning and Dylan’s been in my arms all day,” Jay says. “I’ve got a boy that’s going to carry on our family name and I’m so thrilled to have little Dylan in our lives.”

Jay rushed home to Nashville from a concert stop in Detroit and “went into panic mode” when he found out his wife was in labor.

When baby Dylan was born, Jay says he left Allison on the operating table and returned to Detroit ““ with police escort ““ and performed the sold-out show.

Jay’s happy to be home again with both Dylan and his 19-month-old Madeline, who “leaned over and kissed [Dylan] on the cheek” when she first saw him.

“It was one of those moments I’ll never forget as long as I live,” Jay says.

He’s back on the road with the Flatts this weekend in St. Louis.