Markov chain input?

Ben L.

So I wrote another markov chain program. It works like every other markov chain program. You feed it a many-line text file with sentences or paragraphs on each line. It spits out random gibberish vaguely resembling the original corpus. Now I just need to find a corpus.

Currently, I have these two. I considered trying Google's ngrams corpus, but it turns out I don't have enough hard drive space to hold the compressed files, let alone memory to hold the graph my program generates.

I'm tempted to use these forums as a corpus, but I don't have the data from last time, and it was basically just the page stripped of html, which made our quoting habit look really weird.

Any ideas?

locallunatic

@Ben L. said:

I'm tempted to use these forums as a corpus, but I don't have the data from last time, and it was basically just the page stripped of html, which made our quoting habit look really weird.

You could do another grab of the forums, but with adjustments to compensate for quoting. I mean just strip the quote during the html strip.

Lorne Kates

@Ben L. said:

Any ideas?

A program that takes disperate piece of information and finally pieces them together into a insanity-inducing whole that cannot truly be comprehended by Man?

[url="http://www.dagonbytes.com/thelibrary/lovecraft/thecallofcthulhu.htm"]DO YOU EVEN NEED TO ASK?[/url]

Ben L.

@Lorne Kates said:

@Ben L. said:
Any ideas?

A program that takes disperate piece of information and finally pieces them together into a insanity-inducing whole that cannot truly be comprehended by Man?
DO YOU EVEN NEED TO ASK?

The paragraphs were a bit long, so I split it into sentences. Here's 100 lines of output.

"Legrasse for special mentnon; though rather than he proved distractingly imaginative.
This data, received the sculptor's dream or pedestal covered the mad from the sound the spirit of Inspector Legrasse and drawing mystified conclusions.
The notes in measureless aeons even the second of the Greenland coast had come.
Here were horizontal, hence perhaps an Angarola could question without warning upon hearing, after a mental disorder.
As it was a cult ready for Them.
One Survivor and hieroglyphs.
At the cult to the dream-place he was nightmare cult in the slimy stones or mineralogy.
There was only it had taken something frightfully suggestive of ten scaffolds set sail in lonely places.
But at no more.
I do not feared the haunted wood.
I live, I made me till it obscured such as puzzled as chanted aloud: "Ph'nglui mglw'nafh Cthulhu still lives, too, seemed distorted horrors beyond good and attended to be told.
When, after a repetition of R'lyeh, preserved in corroboration and from the number being slightly ill-mannered, which they ploughed on which could see the whole which he was of strange parallelisms and so steep a total effacing of a venomous seething astern; where - God knows the eccentric sculptor had cried out of the key till death of an architect went so unheard-of.
S.
There had always would exist, hidden in ballast, had found.
I told the passable road they lost no more.
It seemed highly out of her noisome crew of strange visitations.
Its gelatinous green ooze and authentic.
The Emma's crew began at the various persons covering the mate's regret as the aurora leaped high up by a repetition of his hands.
Theosophists have heaved up at the stars which I shall ever be the wooded swamps south of my uncle had largely alluded to be confronted by white men.
Out of her course by the strange cults or image, and anthropological source-books as follows:
I know it.
But it a sprinkling of her bombardment and then only imagination had come to a chain.
One thing I could not preserved, but in Tow.
March 1st.
There were right or somehow propelled himself working, chilled and hidden in opening it, though the historic affiliations in the main document was unmistakable; yet centuries and an angle of the ring of them.
But just now the almost gave me when the six of 1926.
No man and Miss Murray's Witch-Cult in prose, and unmistakably akin to find the grip of dank stones or system of the malevolent tom-tom had lapsed suddenly played upon the distant wastes and extended a placid island of March 22nd, and even torture could do not a mere chance which had not live.
A weird bunch of the mountains of serious native superstition.
It was voodoo, apparently, had been captured members, absolutely alone among them reeled, one day he has to give me till his feet, and he thinks two severely wounded ones were seamen, and of dreams? Was I made their kindred idols brought in the sounds frequently been studying the flowers of Negroes and urgent demand, Inspector Legrasse about one man and always existed and slowly, the same utter strangeness of masonry which never spoken aloud, only half-heartedly that young Wilcox was intensified by an unknown island and met and it is echoed in halting English that is, one night clothes, when the sculptor and scientist agreed on hind and phantasies which chills me pray that, if it of having in archaeology.
The chief of bobbing lanterns.
Examining one of course, was a very crude bas-relief of earth, and April 18, 1925.
This manuscript told their mastery of sculpture at its gist enough to the cult, of ideas it climbing if only echoes which sailed for enlightenment was built in its hateful original form, whilst the various persons covering the heart, induced by day he might the mass motion being immeasurably the original of Dr.
Once before, in Princeton University, and of the polarising miasma welling out in recognizing both hieroglyphics of the Newport boat; falling suddenly; as Cyclopean masonry which gave an admission of his old age, Wilcox of time some interest; for the sea in aspect, he knew how.
Even the inner walls as l wish him mad.
The Emma's crew an excellent family slightly known of hurried cabling and the group of group of the men, and shadow of the statuette, idol, fetish, or flee from this part of evident pictorial intent, though there be easy to his art profoundly, and poor Johansen reached the key to April 2nd, From February 28 to the cult-members who had been resorted to him, and hear with no reason to see with Them."
But I carefully noted by the old Castro continued, were told it proved doubly exciting to possess it.
Three men formed a loss to civilisation's youth - and worthy man.
It was, aside and was then only a learned friend had learned and copied an admission of April 18, 1925.
After vigintillions of her course of voodooism to fire savagely and impious with anything but heavily armed steam was acute, but his hordes, hidden cults, with strange visitations.
Professor Angell.
I shall go.
A dispatch from a wide affiliations in America, where a guide.
Three men formed themselves to be; though scattered notes of fighting.
The most sensible conclusions.
Johansen reached a venomous seething astern; where I read, and mongrel Louisianans?.
That is well by his curiosity in lonely places.
At length the period as he began in area; obviously of an excellent reputation, and of policemen.
Death would be in the mountainous white polypous thing - for any visible disorder, but in Boston.
At length the professor, too common for data.
It had done by an exploding bladder, a knowledge of our world to its origin and cannot think! Let me to recall and twisted menace had from this or not but formless white bulk beyond the giant monster's malign and muffled tom-toms.
One case, which tried to kill myself forthwith.
This man, and and carried in the jamb and which I saw no other eye.
Wilcox afterwards said, after the time and who came out from researches into the vice-admiralty.
The rest in the last.
From a part of the doubled-up, crouching hind legs gripped the West Greenland tale of which would cease, and was so singular clay bas-relief on a second mate Johansen proceeded to the hellish outlines had failed to revive Them from the unclean froth like an outside air of his vessel head downward, the same period as "Christiana." I discovered, lay in his quarters revealed that the odd cuttings largely West Greenland tale of the centre hung, head on the cowed squatters who had from an old number being forced to have broken loose again, each straining in delirium.
A time and Miss Murray's Witch-Cult in a giant shape was always be later met and landed at the old man's subsidence into the dreams he did not meant only moderate length.
The two sounds in a secretary.
All denied a narrow hill street leading up by the manuscript told an attic window after the truth when the trim wharves in sight; and ooze and human abnormality than he told by my host adieu and one of the squatters who had influenced his entire set up from some obscure sort of that preserved by the revelation or otherwise.
Geologists, the dream-sculpture of the ingrained skepticism then that spread fear I do not try to liberate Their great wings, and wish it had never leave me when It was fully as a prominent museums; so singular tribe or not put caution before audacity and anthropological notes gave a diseased fancy could blot out at the fetish which I live, I suspected a short time I could have been only the document I believe, some peculiarly heavy storms and the Thayer Street all dripping Babylon of course, was nothing familiar to adopt what Professor Angell had not put a sober and some interest; for some years; and well, finding it appears, they danced when the ground were brief trip to my life will institute an excellent reputation, and inconclusive questioning felt that gathering of summer must not be found subconscious residuum had come down from outside air of an inch thick and whilst a great cities.
I had found.
If heaven seemed highly characteristic of the date the intensity the vast, loathsome faith.
This person was indeed to him, were noted and perspective seemed instinct with an old papers and in S.
What, in height, and Angstrom.
He will, I tottering cities of them.
What of the back and frenzy.
The two perished of the whole matter and drawing mystified conclusions.
And now, but voodoo of human sacrifices there came at Tulane University could have varied; but I do not preserved, but what seemed recorded troubles in silence through pale undergrowth beyond man's power to my great-uncle's heir and brought out, and desperate though there were in 1908, seventeen years before, it occurred to examine the court of those rendered only this: "In his fellows, and had animated this dictum, but when I almost material.
Professor Angell bearing the case; calling from outside cellar-door.
His rule of summer must be pieced together again.
A.
Then came nearly up from the scorn of the document away and which lurks in his pocket.
The bas-relief was from his entire meeting, was then convinced the ocassion of really awed by Esquimaux diabolists and cuttings collected by a secret - so that accursed city ceased, and evidence that was very place the inability of voodooism to gaze at strange story of evident pictorial intent, though the bowsprit of the terrible to remember as befitted one of strange dreaming had he knew too much, in this part of dissociated knowledge in the lovely olonial houses in this earth, and occultism, went mad, and weedy Cyclopean masonry which he had not enlarge his visitor, especially those descriptive of having been there is the speculations of the yacht's deck, and in the front amidst the work in Louisiana, and They and the professor's manuscript, the masonry which my uncle?
Donovan felt the bas-relief.
But of even the rebuke I mention occurs in Boston.
I.", and older matters mentioned only thus could possibly have stopped my uncle.
Young Wilcox's malady suddenly ceased.
The bas-relief so vast.
March 22.
Meanwhile no more.
I suppose, again when other side of common pattern.
But before anything more affected, though its mounting steam.
Collins and at Sydney, and which only as I now and more be surprised to be long.
Mtg.
So a rear room of the manuscript continued, Wilcox had from shots were insane on which faint beating of the greater and sending out of myth-maker and mulattoes, largely West Indians or because of a week.
There was a tour of files and weedy Cyclopean city dismissed him one when It may sink, and will be found him one of summer must have been resorted to find himself working, chilled and disabled but were carried away on a secret societies and redundance, but archeology.
These Great Old Castro continued, were arts which the central idea of morbid habitation a detailed confirmation of shining eyes and was not tell its wharf in heaven! - vainly, as they said, "It is why the reason to civilisation's youth of all felt sure that shape and unnatural malignancy, was a daemon galleon.
Briden and untraversed by an octopus-like head surmounted a placid island and I now faintly audible far, far within his family in clay and sinister with anything but formless nocturnal impressions appear here and hieroglyphs.
Totally separate and about foot of the stars were brief trip these designs seemed to literary style.
At the cowed squatters refused point-blank to beasts; and nearly hypnotised with foreign parts; and tolerably dry.
I.", and irrelevant accounts of the first part in those crazily elusive angles and what deep and laid it a theosophist colony as given, ran the police a positive quality; for a week of a prelude to Sydney and air of that preserved by a bundle of ancient aeons when the end Legrasse had his men, and anthropological notes formed a widely south of great wings, and what an investigation of Odysseus.
With him he must, at first-hand, though my uncle kept no more subtly fearful because he was delayed and a bland optimism.
The rest followed him, who lived ages before men shouting monotonously in which walked or else will one else seemed to the tainted outside air of shrieking and see if there is described as it must have received with strange dreaming had his kind, he could be the sky; but I made a raid on unguided into its wharf in society and then mankind as might help them in the creatures held no pretense to churn the bas-relief was nothing familiar to the most shockingly frightful.
I have employed a somewhat unkempt in S.
Examined at 1908 A.
Galvez, I now entered by his equally weird bunch of theosophists and the members severally shook their ordering back and the pertinent answers came, the extravagance of April storm; but some Runic inscriptions which gave a darkness almost a very innocent sailors at once more than he had terrified the young world and frenzy.
Once before, when I later met with his offering created.
It read the rationalism of the thoughts must serve to dress in a dream or February 20th with a centre of truth, flashed out from the shrunken and suggestion; for, although there be screaming with his hordes, hidden cults, with laws and had begun its origin, apart from the stars had visions he spoke in joy.
something frightfully suggestive of nocturnal impressions appear here and the lethal waters; whilst within his companion, died.
The older than the habit of the first to the centre hung, head surmounted a hidden cults, with the deep, and at intervals when viewed it appears, Professor Webb had from this dripping with the vice-admiralty.
But at a hideous a tour of the secret - surfaces too much, in 1925 had formerly dreamed, but archeology.

Lorne Kates

@Ben L. said:

S.

That's some mighty fine Markoving there, Ben.

pjt33

COCA or BNC? They're intended to be corpora for serious research, but that doesn't mean that they can't be put to silly purposes.

dhromed

@Lorne Kates said:

@Ben L. said:
S.

That's some mighty fine Markoving there, Ben.

Markov Chain link