That's a bit of an oversimplification. In the Reformed church (and most other churches will have a similar doctrine, maybe with words moved around a little) the idea is that a true belief - that is, a belief that will result in being saved from Hell - will result in worship, church attendance, etcetera.
I think I just ninja'd you
The other side of it is that admitting you've done wrong isn't enough. You have to try to change things, or "make up for it", to use the vernacular. This most certainly is not intended to allow for a mindless "I murdered thirty guys, but I told God I was sorry so it's all good".
In short, forgiveness is not a justification for wrongdoing.
While it's great to try to do good, the bible says that man cannot do good by sheer will alone. Even after believing.
Romans 3:23, "For all have sinned and fall short of the glory of God."
James 2:10, "For whoever keeps the whole law and yet stumbles at just one point is guilty of breaking all of it."
(I can reference several more if you want
Instead, the only way is to believe in Jesus to have him take away your sins.
Roamns 6:23, "For the wages of sin is death, but the gift of God is eternal life in Christ Jesus our Lord."
Acts 16: 29-31, "29 The jailer called for lights, rushed in and fell trembling before Paul and Silas. 30 He then brought them out and asked, “Sirs, what must I do to be saved?” 31 They replied, “Believe in the Lord Jesus, and you will be saved—you and your household.”
(I can reference several more if you want.)