Lack of a clear goal for the characters isn't necessarily the same thing as saying plotless. That's what's different. Grimgar has a plot progression, with specific events. SoL almost entirely lack that.
But let me clarify exactly what I mean here: I mean that SoL series have an extremely weak "arrow of time". For instance, K-On only really has events that matter right at the start and right at the end, so they set it up, and wind it down, but the episodes in the middle could be viewed in randomized order, and you'd barely notice. The only caveat is when such series decide to add a new character to the mix, but most of them introduce the full cast in at most 4-5 episodes then stabilize things completely for the rest of the series. There might be the occasional running gag, e.g. a scene in one episode might reference a scene in another, but most scenes could be randomized without any story problems arising.
Grimgar is different, because it has a "direction of time" arrow. Even when they're having idle chats, eating or relaxing, there is dialogue informed by what happened before. Basically if you would be genuinely worried about "spoilers" if someone watched the episodes completely out of order it's not really a slice of life series, as there's already a perfectly good term for that: drama. If "spoilers" would be a big problem if the show is watched "out of order" then it's not really SoL.
But SoL can be used, in the same sense that comedy can be used. e.g. Star Wars has comedy, but it's no "A comedy". Sure, Grimgar could use "slice of life". But I don't think it qualifies. It has emotional moments, but they reveal things about the character's past, and they build on each other. They're not the same type of standalone episodic events that Slice of Life is normally referring to.