I recall a book (no idea of the title, or publication date (<80s, certainly), but never mind) in which a kind of secret deep-sea "Captain Nemo"-like civilisation of scientists was discovered by the central character, previously surface-based. (Or possibly he got too close and they effectively kidnapped him from his bathysphere.) To survive at depth, hyperoxygenated liquids were used in order to fill the lungs and allow high-pressure living, the breathing reflexes were surgically suspended and as part of this there were no longer any ways of speaking.
This (self-styled) rationalist society therefore resorted to non-verbal communication and had got to the stage of abandoning speech, even in written form, in favour of a "circuit diagram"-like written representation of their typically complex ideas in a non-linear manner. It was a "different for the sake of difference" concept, of course. A direct counterpart to the unthinkingly unimaginative trope which gives us "All aliens speak English". It was frankly silly, I suppose (to be honest, it had very little importance as far as plot) but it did occur to me as an interesting tale to retell, in context.
Back to the real world, deaf persons often learn signing very well (especially those deaf from birth), blind people learn Braille in lieu of normal writing (but see below) and deaf-blind individuals can learn their communicate with others through a suitably tactile variant of signing. If starting more or less from birth, I suspect that almost any non-verbal communication method could be as effectively adopted as a "mother tongue", limited only by the parents' (or others') ability to themselves gain proficiency in converse, given their life-long habituation towards their more traditional methods.
As to the alternate methods of representation for punctuation, look at Morse. The difference between letters, numbers, punctuation and 'meta characters' is (to the untrained eye/ear) negligible[1]. Dots and dashes, in whatever form they are conveyed, can represent such subtleties, of course, with a combination allocated to the desired item. The same is true of Braille dots (which I do not know, yet am invariably drawn to examining such signs as I see in various buildings that I might visit and have them placed around[2], when I have leisure to do so), and not only do I not know enough to visually discern them (except by painstaking inspection and inference, each and every time), but I've tried to read them in a purely tactile manner[3] and just do not have the sensitivity.
And yet I am fully aware that the sightless regularly use various 'reader technologies' that are even finer (and dynamic) than these public signs, and some (like Peter White of the BBC, IIRC, or it might have been someone he was talking to about this, in a radio programme celebrating a special anniversary of BBC radio for the blind, a month or two ago) can read their scripts far faster than they are reading out loud, to rival many sighted people's actual ability with regular writing.
Not that any of this necessarily suggests a direction for the OP's requested language formation, but it justifies efforts in all kinds of directions for those that might wish to make them.
[1] There's logic to the patterns, I know, but I've no idea what. And compare a "!" with an "i" in the regularly-written alphabet, with an eye unsullied by your existing knowledge, the basic "1" and san-serif "I" can confuse where context isn't clear, as well as such similarities of shape similarities between "5" and "S" and other combinations that help compose 1337-5p34k practitioners' communiques... 17 (/\|\| |33 707411y |\|()|\|53|V51|3|_3, `/37 |234|)4|3|_3!!!111!!oneoneone111!!!
[2] There's one place where it's obvious the sign says "Please press 4 help", instead of the "...for..." of the visually written part of the sign. Then there's even a sign outside the WC (separately labelled as such) labelling "toilet light switch". Which I always think is very useful for a blind person. (There's no other signs on any other switches, so it can't be to avoid confusion with door-lock switches, emergency fire points, etc.)
[3] Already knowing where to feel, and I always wonder if there's a sign convention that allows a blind-person to zero in on such signs, or at least to rule out the possibility that (having found no signs) there is nothing just beyond their explored environment that they should have felt for.