[docs] Add switch commitment documentation (#2526)

* remove references to no-longer existing switch commitment hash

(as switch commitments were removed in ca8447f3bd
and moved into the blinding factor of the Pedersen Commitment)

* some rewording (points vs curves) and fix of small formatting issues

* Add switch commitment documentation

core/src/core/transaction.rs

@@ -1226,10 +1226,7 @@ impl Readable for OutputFeatures {
 /// Output for a transaction, defining the new ownership of coins that are being
 /// transferred. The commitment is a blinded value for the output while the
 /// range proof guarantees the commitment includes a positive value without
-/// overflow and the ownership of the private key. The switch commitment hash
+/// overflow and the ownership of the private key.
-/// provides future-proofing against quantum-based attacks, as well as providing
-/// wallet implementations with a way to identify their outputs for wallet
-/// reconstruction.
 #[derive(Debug, Copy, Clone, Serialize, Deserialize)]
 pub struct Output {
 	/// Options for an output's structure or use

doc/coinbase_maturity.md

@@ -119,9 +119,7 @@ To summarize -
 
 Output MMR stores output hashes based on `commitment|features` (the commitment itself is not sufficient).
 
-We do not need to include the range proof or the switch commitment hash in the generation of the output hash.
+We do not need to include the range proof in the generation of the output hash.
-
-We do not need to encode the lock height in the switch commitment hash.
 
 To spend an output we continue to need -
 

doc/intro.md

@@ -90,7 +90,7 @@ fundamental properties are achieved.
 Building upon the properties of ECC we described above, one can obscure the values
 in a transaction.
 
-If _v_ is the value of a transaction input or output and _H_ an elliptic curve, we can simply
+If _v_ is the value of a transaction input or output and _H_ a point on the elliptic curve _C_, we can simply 
 embed `v*H` instead of _v_ in a transaction. This works because using the ECC
 operations, we can still validate that the sum of the outputs of a transaction equals the
 sum of inputs:
@@ -99,11 +99,12 @@ sum of inputs:
 
 Verifying this property on every transaction allows the protocol to verify that a
 transaction doesn't create money out of thin air, without knowing what the actual
-values are. However, there are a finite number of usable values and one could try every single
+values are. However, there are a finite number of usable values (transaction amounts) and one 
+could try every single 
 one of them to guess the value of your transaction. In addition, knowing v1 (from
 a previous transaction for example) and the resulting `v1*H` reveals all outputs with
-value v1 across the blockchain. For these reasons, we introduce a second elliptic curve
+value v1 across the blockchain. For these reasons, we introduce a second point _G_ on the same elliptic curve
-_G_ (practically _G_ is just another generator point on the same curve group as _H_) and
+(practically _G_ is just another generator point on the same curve group as _H_) and
 a private key _r_ used as a *blinding factor*.
 
 An input or output value in a transaction can then be expressed as:
@@ -112,9 +113,10 @@ An input or output value in a transaction can then be expressed as:
 
 Where:
 
-* _r_ is a private key used as a blinding factor, _G_ is an elliptic curve and
+* _r_ is a private key used as a blinding factor, _G_ is a point on the elliptic curve _C_ and
-  their product `r*G` is the public key for _r_ on _G_.
+  their product `r*G` is the public key for _r_ (using _G_ as generator point).
-* _v_ is the value of an input or output and _H_ is another elliptic curve.
+* _v_ is the value of an input or output and _H_ is another point on the elliptic curve _C_,
+  together producing another public key `v*H` (using _H_ as generator point).
 
 Neither _v_ nor _r_ can be deduced, leveraging the fundamental properties of Elliptic
 Curve Cryptography. `r*G + v*H` is called a _Pedersen Commitment_.
@@ -122,8 +124,8 @@ Curve Cryptography. `r*G + v*H` is called a _Pedersen Commitment_.
 As an example, let's assume we want to build a transaction with two inputs and one
 output. We have (ignoring fees):
 
-* vi1 and vi2 as input values.
+* `vi1` and `vi2` as input values.
-* vo3 as output value.
+* `vo3` as output value.
 
 Such that:
 
@@ -143,8 +145,9 @@ transaction can be done without knowing any of the values.
 
 As a final note, this idea is actually derived from Greg Maxwell's
 [Confidential Transactions](https://elementsproject.org/features/confidential-transactions/investigation),
-which is itself derived from an Adam Back proposal for homomorphic values applied
+which is itself derived from an 
-to Bitcoin.
+[Adam Back proposal for homomorphic values](https://bitcointalk.org/index.php?topic=305791.0) 
+applied to Bitcoin.
 
 #### Ownership
 
@@ -168,7 +171,7 @@ You need to build a simple transaction such that:
 
     Xi => Y
 
-Where _Xi_ is an input that spends your _X_ output and Y is Carol's output. There is no way to build
+Where _Xi_ is an input that spends your _X_ output and _Y_ is Carol's output. There is no way to build
 such a transaction and balance it without knowing your private key of 28. Indeed, if Carol
 is to balance this transaction, she needs to know both the value sent and your private key
 so that:
@@ -190,14 +193,19 @@ She picks 113 say, and what ends up on the blockchain is:
 Now the transaction no longer sums to zero and we have an _excess value_ on _G_
 (85), which is the result of the summation of all blinding factors. But because `85*G` is
 a valid public key on the elliptic curve _G_, with private key 85,
-for any x and y, only if `y = 0` is `x*G + y*H` a valid public key on _G_.
+for any x and y, only if `y = 0` is `x*G + y*H` a valid public key on the elliptic curve 
+using generator point _G_.
 
-So all the protocol needs to verify is that (`Y - Xi`) is a valid public key on _G_ and that
+So all the protocol needs to verify is that (`Y - Xi`) is a valid public key on the curve 
-the transacting parties collectively know the private key (85 in our transaction with Carol). The
+and that the transacting parties collectively know the private key `x` (85 in our transaction with
-simplest way to do so is to require a signature built with the excess value (85),
+Carol) of this public key. If they can prove that they know the private key to `x*G + y*H` using
+generator point _G_ then this proves that `y` must be `0` (meaning above that the sum of all 
+inputs and outputs equals `0`).
+
+The simplest way to do so is to require a signature built with the excess value (85),
 which then validates that:
 
-* The transacting parties collectively know the private key, and
+* The transacting parties collectively know the private key (the excess value 85), and
 * The sum of the transaction outputs, minus the inputs, sum to a zero value
   (because only a valid public key, matching the private key, will check against
   the signature).
@@ -237,13 +245,13 @@ create new funds in every transaction.
 For example, one could create a transaction with an input of 2 and outputs of 5
 and -3 and still obtain a well-balanced transaction, following the definition in
 the previous sections. This can't be easily detected because even if _x_ is
-negative, the corresponding point `x.H` on the curve looks like any other.
+negative, the corresponding point `x*H` on the curve looks like any other.
 
 To solve this problem, MimbleWimble leverages another cryptographic concept (also
 coming from Confidential Transactions) called
 range proofs: a proof that a number falls within a given range, without revealing
 the number. We won't elaborate on the range proof, but you just need to know
-that for any `r.G + v.H` we can build a proof that will show that _v_ is greater than
+that for any `r*G + v*H` we can build a proof that will show that _v_ is greater than
 zero and does not overflow.
 
 It's also important to note that in order to create a valid range proof from the example above, both of the values 113 and 28 used in creating and signing for the excess value must be known. The reason for this, as well as a more detailed description of range proofs are further detailed in the [range proof paper](https://eprint.iacr.org/2017/1066.pdf).
@@ -255,7 +263,7 @@ A MimbleWimble transaction includes the following:
 * A set of inputs, that reference and spend a set of previous outputs.
 * A set of new outputs that include:
   * A value and a blinding factor (which is just a new private key) multiplied on
-  a curve and summed to be `r.G + v.H`.
+  a curve and summed to be `r*G + v*H`.
   * A range proof that shows that v is non-negative.
 * An explicit transaction fee, in clear.
 * A signature, computed by taking the excess blinding value (the sum of all

doc/pruning.md

@@ -63,8 +63,7 @@ The full validation of the chain state requires that:
 In addition, while not necessary to validate the full chain state, to be able
 to accept and validate new blocks additional data is required:
 
-* The output features and switch commitments, making the full output data
+* The output features, making the full output data necessary for all UTXOs.
-  necessary for all UTXOs.
 
 At minimum, this requires the following data:
 

doc/switch_commitment.md

@@ -0,0 +1,293 @@
+# Introduction to Switch Commitments
+
+## General introduction
+
+In cryptography a _Commitment_ (or _commitment scheme_) refers to a concept which can be imagined
+like a box with a lock. You can put something into the box (for example a piece of a paper with a
+secret number written on it), lock it and give it to another person (or the public).
+
+The other person doesn't know yet what's the secret number in the box, but if you decide to publish
+your secret number later in time and want to prove that this really is the secret which you came
+up with in the first place (and not a different one) you can prove this simply by giving the
+key of the box to the other person.
+
+They can unlock the box, compare the secret within the box with the secret you just published
+and can be sure that you didn't change your secret since you locked it. You "**commited**"
+to the secret number beforehand, meaning you cannot change it between the time of
+commitment and time of revealing.
+
+
+## Examples
+
+### Hash Commitment
+
+A simple commitment scheme can be realized with a cryptographic hash function. For example: Alice and Bob
+want to play _"Guess my number"_ and Alice comes up with with her really secret number `29` which
+Bob has to guess in the game, then before the game starts, Alice calculates:
+
+    hash( 29 + r )
+
+an publishes the result to Bob. The `r` is a randomly chosen  _Blinding Factor_ which is
+needed because otherwise Bob could just try hashing all the possible numbers for the game and
+compare the hashes.
+
+When the game is finished, Alice simply needs to publish her secret number `29` and the
+blinding factor `r` and Bob can calculate the hash himself and easily verify that Alice
+did not change the secret number during the game.
+
+
+### Pedersen Commitment
+
+Other, more advanced commitment schemes can have additional properties. For example MimbleWimble
+and Confidential Transactions (CT) make heavy use of
+_[Pedersen Commitments](https://link.springer.com/content/pdf/10.1007/3-540-46766-1_9.pdf)_,
+which are _homomorphic_ commitments. Homomorphic in this context means that (speaking in the
+"box" metaphor from above) you can take two of these locked boxes (_box1_ and _box2_) and
+somehow "_add_" them together, so that you
+get a single box as result (which still is locked), and if you open this single box later
+(like in the examples before) the secret it contains, is the sum of the secrets
+from _box1_ and _box2_.
+
+While this "box" metaphor no longer seems to be reasonable in the real-world this
+is perfectly possible using the properties of operations on elliptic curves.   
+
+Look into [Introduction to MimbleWimble](intro.md) for further details on Pedersen Commitments
+and how they are used in Grin.
+
+
+## Properties of commitment schemes:
+
+In general for any commitment scheme we can identify two important properties
+which can be weaker or stronger, depending on the type of commitment scheme:
+
+- **Hidingness (or Confidentiality):** How good is the commitment scheme protecting the secret
+  commitment. Or speaking in terms of our example from above: what would an attacker need to
+  open the box (and learn the secret number) without having the key to unlock it?
+ 
+- **Bindingness:** Is it possible at all (or how hard would it be) for an attacker to somehow
+  find a different secret, which would produce the same commitment, so that the attacker could
+  later open the commitment to a different secret, thus breaking the _binding_ of the
+  commitment.
+ 
+### Security of these properties:
+
+For these two properties different security levels can be identified.
+
+The two most important combinations of these are
+
+- **perfectly binding** and **computationally hiding** commitment schemes and
+- **computationally binding** and **perfectly hiding** commitment schemes
+
+"_Computationally_" binding or hiding means that the property (bindingness/hidingness)
+is secured by the fact that the underlying mathematical problem is too hard to be solved
+with existing computing power in reasonable time (i.e. not breakable today as computational
+resources are bound in the real world).
+
+"_Perfectly_" binding or hiding means that even with infinite computing power
+it would be impossible to break the property (bindingness/hidingness).
+
+
+
+### Mutual exclusivity:
+
+It is important to realize that it's **impossible** that any commitment scheme can be
+_perfectly binding_ **and** _perfectly hiding_ at the same time. This can be easily shown
+with a thought experiment: Imagine an attacker having infinite computing power, he could
+simply generate a commitment for all possible values (and blinding factors) until finding a
+pair that outputs the same commitment. If we further assume the commitment scheme is
+_perfectly binding_ (meaning there cannot be two different values leading to the same
+commitment) this uniquely would identify the value within the commitment, thus
+breaking the hidingness.
+
+The same is true the other way around. If a commitment scheme is _perfectly hiding_
+there must exist several input values resulting in the same commitment (otherwise an
+attacker with infinite computing power could just try all possible values as
+described above). This concludes that the commitment scheme cannot be _perfectly
+binding_.
+
+#### Always a compromise
+
+The key take-away point is this: it's **always a compromise**, you can never have both
+properties (_hidingness_ and _bindingness_) with _perfect_ security. If one is _perfectly_
+secure then the other can be at most _computationally_ secure
+(and the other way around).
+
+
+### Considerations for crytpocurrencies:
+
+Which roles play these properties in the design of cryptocurrencies?
+
+**Hidingness**:
+In privacy oriented cryptocurrencies like Grin commitment schemes are used to secure
+the contents of transactions. The sender commits to an amount of coins he sends, but for
+the general public the concrete amount (and also the information who the sender is) should
+remain private (protected by the _hidingness_ property of the commitment scheme).
+
+**Bindingness**:
+At the same time no transaction creator should ever be able to change his commitment
+to a different transaction amount later in time. (If this would be possible, an attacker
+could spend more coins than previously were commited to in an UTXO (unspent transaction
+output) and therefore inflate coins out of thin air. Even worse, as the amounts are
+hidden, this could happen undetected.)
+
+So there is a valid interest in having both of these properties always secured and
+never be violated.
+
+Even with the intent being that both of these properties will hold for the lifetime
+of a cryptocurrency, still a choice has to be made which commitment scheme to use.
+
+
+#### A hard choice?
+
+Which one of these two properties needs to be _perfectly_ safe
+and for which one it would be sufficient to be _computationally_ safe?
+Or in other words: in case of disaster, if the commitment scheme unexpectedly
+gets broken, which one of the two properties should be valued higher?
+Economical soundness (no hidden inflation possible) or ensured privacy (privacy will
+be preserved)?
+
+This seems like a hard to choice to make.
+
+
+If we look closer into this we realize that the commitment scheme only needs to be
+_perfectly_ binding at the point in time when the scheme actually gets broken. Until
+then it will be safe even if it's only _computationally_ binding.
+
+At the same time a privacy-oriented cryptocurrency needs to ensure the _hidingness_
+property **forever**. Unlike the _binding_ property (which only is important at the
+time when a transaction is created and will not affect past transactions) the _hidingness_
+property must be ensured for all times, because in the unfortunate case should the
+commitment scheme be broken, an attacker could go back in the chain and unblind
+past transactions thus breaking the privacy property retroactively.
+
+
+## Properties of Pedersen Commitments
+
+Pedersen Commitments are **computationally binding** and **perfectly hiding** as for a given
+commitment to the value "v": `v*H + r*G` there may exist a pair of different values `r1`
+and `v1` such that the sum will be the same. Even if you have infinite computing power
+and could try all possible values, you would not be able to tell which one is the original one
+(thus _perfectly hiding_).
+
+
+## Introducing Switch Commitments
+
+So what could be done if the bindingness of the Pedersen Commitment unexpectedly gets broken?
+
+In general a cryptocurrency confronted with a broken commitment scheme could choose to
+change the scheme in use, but the problem with this approach would be that it requires to
+create new transaction outputs using the new scheme to make funds secure again. This would
+require that every coin holder has to move his coins into a new transaction ouptut.
+If coins are not moved into a "new" transaction they would not profit from the
+security of the new commitment scheme. Also this has to happen **before** the scheme gets
+actually broken in the wild, as otherwise the existing UTXOs no longer can be assumed
+to contain correct values.
+
+In this situation [_Switch Commitments_](https://eprint.iacr.org/2017/237.pdf) offer a neat 
+solution. These type of commitments allow changing the properties of the commitments just
+by changing the revealing / validating procedure without changing the way how commitments
+are created. (You "_switch_" to a new validation scheme and it automatically works backwards
+compatible with commitments created long before the actual "_switch_").
+
+
+### How does this work in detail
+
+First let's introduce a new commitment scheme: The **ElGamal commitment** scheme is a commitment
+scheme similiar to Pedersen Commitments and it's _perfectly binding_ (but only _computationally
+hiding_ as we can never have both).
+Compared to a Pedersen Commitment it looks very similiar, it just adds an additional
+element, calculated by multiplying the blinding value `r` with another generator point `J`:
+
+    v*H + r*G ,  r*J
+
+So if we store the additional field `r*J` and ignore it for now, we can treat it as
+Peterson Commitments like before (until we decide to also validate the full ElGamal
+commitment at some time in future). This is exactly what was implemented in an
+[earlier version of Grin](https://github.com/mimblewimble/grin/blob/5a47a1710112153fb38e4406251c9874c366f1c0/core/src/core/transaction.rs#L812)
+(before mainnet was launched). In detail: the hashed value of `r*J`
+(_switch\_commit\_hash_) was added to the transaction output, but this came with
+the burden of increasing the size of each output by 32 bytes.
+
+Fortunately, later on the Mimblewimble mailinglist Tim Ruffing came up with a really
+[beautiful idea](https://lists.launchpad.net/mimblewimble/msg00479.html)
+(initially suggested by Pieter Wuille), which offers the same advantages but doesn't
+need this extra storage of an additional element per transaction output:
+
+The idea is the following:
+
+A normal Pedersen commitment looks like this:
+
+    v*H + r*G
+
+(`v` is value of the input/output, `r` is a truly random blinding factor and `H` and `G` are
+two generator points on the elliptic curve).
+
+If we adapt this by having `r` not being random itself, but using another random number `r'`
+and create the Pedersen Commitment:
+
+    v*H + r*G
+
+such that:
+
+    r = r' + hash( v*H + r'*G  ,  r'*J )
+   
+(using the additional third generation point `J` on the curve) then `r` still is perfectly
+valid as a blinding factor (as it's still randomly distributed) but now we see
+that the part within the brackets of the hash function (`v*H + r'*G  ,  r'*J`) is an
+**ElGamal commitment**.
+
+This neat idea lead to the removal of the switch commitment hash from the outputs in this
+(and following) [pull requests](https://github.com/mimblewimble/grin/issues/998) as now it
+could be easily included into the Pedersen Commitments.
+
+
+This is how it is currently implemented in Grin. Pedersen commitments are
+used for the Confidential Transaction but instead of choosing the blinding factor `r`
+only by random, it is calculated by adding the hash of an ElGamal commitment to a random `r'`
+(see here in [main_impl.h#L267](https://github.com/mimblewimble/secp256k1-zkp/blob/73617d0fcc4f51896cce4f9a1a6977a6958297f8/src/modules/commitment/main_impl.h#L267)).
+
+
+In general switch commitments first have been described in the paper
+["Switch Commitments: A Safety Switch for Confidential Transactions"](https://eprint.iacr.org/2017/237.pdf)).
+The **"switch"** in the name comes from the fact that you can virtually flip a "switch" in
+the future and simply by changing the validation procedure you can change the strength of
+the bindingness and hidningness property of your commitments and this even works in a
+backwards compatible way with commitments created today.
+
+
+
+## Conclusion:
+
+Grin uses Pedersen Commitments - like other privacy cryptocurrencies do as well - with
+the only difference that the random Blinding factor `r` is created using the ElGamal
+commitment scheme.
+
+This might not seem like a big change on the first look, but this provides an
+important safety measure:
+
+Pedersen Commitments are already _perfectly hiding_ so whatever happens, privacy will
+never be at risk without requiring any action from users. But in case of disaster if the
+bindingness of the commitment scheme gets broken, then switch commitments can be enabled 
+(via a soft fork) requiring that all new transactions prove that their commitment is not
+breaking the bindingness by validating the full ElGamal commitment. 
+
+But in this case users would still have a choice:
+
+- they can decide to continue to create new transations, even if this might compromise 
+  their privacy (only on their **last** UTXOs) as the ElGamal commitment scheme is 
+  only computationally hiding, but at least users would still have access to their coins
+
+- or users can decide to just leave the money alone, walk away and make no more transactions
+  (but preserve their privacy, as their old transactions only validated the Pedersen commitment
+  which is perfectly hiding)
+  
+There are many cases where a privacy leak is much more dangerous to one's life than
+some cryptocurrency might be worth. But this is a decision that should be up to 
+the individual user and switch commitments enable this type of choice.
+
+It also should be made clear that this is a last measure meant for the case of
+disaster and not expected to be needed on the normal path of development. If advances in
+computing would put the hardness of the discrete log problem in question also a lot of
+other cryptographic systems (and cryptocurrencies) will be in urgent need of updating their
+primitives to a future-proof system. The switch commitments just provide an additional layer
+of security if the bindingness of Pedersen commitments breaks suddenly and unexpected.