Anarcho-Capitalist Legal System

Here is how private security and justice might work in the free market. Since free market solutions are organic and evolutionary, this scenario might not be complete, but it shows that reasonable free market solutions can work.
.
Aggression against an individual, or having their contractual agreements violated, or being defrauded, are inherently unpredictable risks. In other words, we cannot in general predict when we will be robbed, otherwise we would arrange not to be in that palce at that time. We cannot in general predict who will break their contracts with us, else we would not contract with them. We cannot in general predict who is defrauding us, else would would not deal with them.
.
These risks are insurable risks because they are unpredictable. The frequency of some risks can be affected by our decisions, such as choosing to live in a better neighborhood versus a crime ridden neighborhood, but we cannot predict anything about individual events. People who fall into the same risk pool pool their money in the form of insurance premiums to insure themselves against unpredictable losses.
.
There are also uninsurable risks. You cannot insure yourself against the risk of suicide. You can easily influence and predict such events. An insurer that provided such a policy would attract suicidal individuals who wanted to leave money to their family and the insurer would rapidly go broke. Similarly you cannot insure yourself against robbing someone else.
.
Since aggression, breaking of contracts, and fraud against individuals are insurable risks, the natural model for resolving such matters is an insurance model. Insurance companies are numerous and large. They indemnify tens of billions if not hundreds of billions of dollars in assets. They do this by investing premium dollars in diversified appreciating hard assets like real estate, which are also the source of their profits. In a security insurance model, individuals purchase polices that indemnify themselves and their property from loss due to aggression, broken contracts, and fraud. Their premium is determined by their risk pool. Competition amongst insurers insures that insurance customers are subdivided into smaller and smaller groups of indistinguishable levels of risk, and ensures the lowest possible premiums. For example if Company A placed desk jockeys in the same risk pool as pro football players for getting injured, the premiums of the desk jockeys would clearly subsidize systematic payments to the football players, whose risk of injury are far higher. Another company that subdivided the groups could offer much lower premiums to the desk jockeys, and they would leave Company A, forcing them to either go broke or raise the premiums for the football players to the natural level that represents their actual risk.
.
When I am aggressed against, I file a claim to recover my losses. Say you steal my car. I file a claim. The company first investigates to make sure I am not defrauding them and discover who actually stole the car. They have an incentive to find out who did it, because regardless, they have to pay for my new car. They want to recover these costs from the actual aggressor.
.
Once they identify you and gather sufficient evidence, they take the case to a reputable arbitrator. They will not simply go to an arbitrator who will always find in their favor, for the judgements of such an arbitrator would simply be ignored. There is no financial incentive for an arbitrator to simply hand out judgements to the highest bidder. Such an arbitrator would quickly gain a reputation for worthless judgements, his judgements would be ignored in the market, and hence there is no incentive to pay for his judgements in the first place. He will rapidly go out of business. This is why Underwriter's Laboratory doesn't just hand out it's seal to any product whose manufacturer pays a fee; if they did the seal would be worthless and they would go out of business post haste.
.
Now, you can report the action against you to your insurance provider, they can perform their own investigation, and they (or you) can take your side of the case to an arbitrator of your choice as well. But again, there is no incentive for him to simply rule in your favor because you pay him. Again a disreputable arbitrator is an out of work arbitrator.
.
In the majority of cases the facts are clear, the investigations will agree, and so will the rulings. Your insurer does not have any incentive to tilt their investigation in your favor, because you haven't insured yourself, indeed you can't, from committing auto theft. It's no skin off their nose if you lose the case.
.
When the judgement comes out against you, you have a couple of choices. You can simply pay up, and be done with it. I would highly advise this cource of action. Because if you don't pay up, my insurer will send a security firm to show up and extract compensation for their loss (they already paid me, remember). They will seize funds or property, or whatever assets are required to fullfil the judgement.
.
Meanwhile, your insurance company won't lift a finger to stop them, because doing so would be extremely costly, and because you can't insure yourself against you committing autotheft. If you try to resist yourself, or by hiring personal security forces, it will be extremely costly and probably get you killed. I guarantee the resources of my insurance company with its hundreds of billions of dollars in holdings will exceed the resources you have at your disposal to resist.
.
Note that my insurer is not initiating force against you. They are simply closing the force transaction that you initiated against me when you stole my car.
.
In the few cases where the facts are not clear enough to produce identical judgements, the two insurers will simply come to a negotiated settlement between themselves, since having security forces "battle it out" to settle the dispute would be incredibly costly.
.
Hence we have a simple insurance based model of criminal justice that incorporates competition amongst insurers, arbitrators, and security firms that does not require a territorial monopoly for any of them. There are clear market mechanisms that result in compliance with reputable arbitration.
.
There are a number of benefits from such a system. For example, since your premium depends on your level of risk, personal responsibility is incentivised. Those who have alarm systems and guns and take other security measures will have lower insurance premiums, and communities are incentivized to expel criminal elements because those elements raise their premiums, just to name a couple.



So, on to the questions:

1. How do you bring suit against someone without legal/security contract(or)?

I don't really understand this question. If I'm understanding it right, I think my answer is that you just file with a reputable arbitrator, who would probably then notify them. I don't see why they would need to have a "legal contractor" ahead of time.

2. Who tries a murder (or other “state” crime) whose victim has no relatives/friends?

This has been discussed many times. I'm going to phrase "Argumentum ad Hobocide" for it: argument by killing hobos. There are a number of simple market possibilities, including but not limited to:

a) The victim's life insurance policy (if he has one) would almost certainly contain a clause providing for the investigation of their death. This would be an obvious market response in the absence of a monopolist of murder investigations for a couple of reasons: people would want their death investigated an the murderer brought to justice, and b) the insurer would want to recoup their loss from the murderer.
b) The murderer's neighbors. Their insurance premiums would certainly go up living next to a murderer, so they would have an incentive to do something about him. His own insurers would probably drop him because of the liability he represents, and his neighbors insurers have an incentive to do something about him because they don't want to pay out on a claim if he kills one of their clients.
c) Charitable foundations. This happens all the time. There are charitable foundations for legal defense, for example. Wealthy patrons who had relatives murdered would endow charitable justice foundations, just like wealthy people who have relatives die from diseases endow medical research and treatment foundations now.

3. How does the legal/security company get paid in question 2?
In the normal ways, given my response to (2). Also, they always accept lower rates or do some work pro bono. This also happens a lot in the legal field. People like justice.

4. Won’t debtor’s prisons eventually be necessary (not necessarily a bad thing) for repeat offenders (destroying someone’s credit may not be enough deterrent for what is ultimately a cash society and contracting out prisoner’s labor will handle question 2.)?

No. Debtor's prisons are a terrible idea because they always operate at a loss. It costs you more to force people to do the work than it does to just right off the debt. Not to mention the fact that I doubt seriously whether forced labor would fly in a libertarian culture, which would have to be in place for free market anarchism to work. This is exactly the contradiction that tore apart the original Democratic party in the 1840s, the clash between the extremely libertarian political philosophy of the time and the existence of slavery, the most unlibertarian of institutions.

Also, there is no reason that a society with a hard currency has to be a "cash society."

5. What prevents a community of X peoples unfairly prosecuting Y peoples as a cheap labor source, in the event of debtor’s prisons being necessary (insert any two ethnic or socioeconomic groups for X and Y)?

It isn't economically feasible to do so in a modern industrialized society. There is a reason that slavery died out peacefully in almost every nation on the Earth (except the US) in the 18th and 19th century. Unskilled forced labor has no hope of competing with the voluntary skilled labor required to run the highly productive machinery of the industrial revolution. This is why slavery had already died out in the industrialized North decades prior to the civil war, and was already dying in the South prior to Lincoln's invasion.