I'm interested in the way some science fiction novels transform language in order to emphasize the 'otherness' of the society being portrayed so that the reader is forced to actively negotiate this difference in the process of understanding the story. We might call this a form of Verfremdung. I'm abusing Brecht's terminology here, but I'm not sure that there's another term that expresses what I mean. Shklovsky's ostranenie is no better, because he -- like Brecht -- is concerned with it primarily as an aesthetic effect, that of making the familiar strange. In the stories I want to look at, something else happens, a form of negotiation between the familiar (self) and the strange (the 'other)', i.e., precisely that which resists being subsumed into the familiar. The emotional distancing, which is so crucial to Brecht's theory, and perhaps implicit in Shklovsky's, is counterproductive here: the story requires a certain amount of identification, of Einfühlung, in order to be most effective. Caryl Emerson speaks of "subject - object relations" in Shklovsky rather than "subject - subject" relations**, which captures the point pretty well.
I'm thinking in particular of Hoban's Riddley Walker and Burgess' A Clockwork Orange, which are narrated in a heavily modified form of English. This is important: instead of interlacing the story with vocabulary in an invented language, they alter the reader's own language. The effect created by this technique is considerably different, I think. Inventing terms in a foreign language marks them clearly as being embedded in a specific context, one which is, moreover, separate from the world of the reader. And by including -- as is so often done -- an explanation or translation of these foreign terms, the 'otherness' of this world is mediated for the reader, the strangeness is in a sense concealed precisely because it is explicitly labelled as foreign. However, when it is our own language that is being distorted nearly to the point of incomprehensibility, we are confronted all the more forcefully by the fact that we cannot take comprehension for granted.
(Both of these stories have first-person narrators, and this, too, may help produce this effect -- the reader is put inside the narrator's head, so to speak, asked to share his point of view. It is not, I think, necessary however.)
One might compare this method with that of those translators who maintain that a translation should not read like an original, that it should maintain a strangeness which reminds the reader that they are reading a translation.
Schleiermacher puts this in a manner which is useful in this context: Entweder der Übersetzer läßt den Schriftsteller möglichst in Ruhe, und bewegt den Leser ihm entgegen [this is our 'faithful' translation]; oder er läßt den Leser möglichst in Ruhe und bewegt den Schriftsteller ihm entgegen [i.e., a naturalizing translation. (From: Über die verschiedenen Methoden des Übersetzens, 1813)
** See Emerson, Caryl. "Shklovsky's ostranenie, Bakhtin's vnenakhodimost' (How Distance Serves an Aesthetics of Arousal Differently from an Aesthetics Based on Pain)". Poetics Today 26.4 (2005) : 637-664.