View source for Karmarkar's algorithm

'''Karmarkar's algorithm''' is an [[algorithm]] introduced by [[Narendra Karmarkar]] in 1984 for solving [[linear programming]] problems. It was the first reasonably efficient algorithm that solves these problems in [[polynomial time]]. The [[ellipsoid method]] is also polynomial time but proved to be inefficient in practice.

Denoting <math>n</math> as the number of variables and <math>L</math> as the number of bits of input to the algorithm, Karmarkar's algorithm requires <math>O(n^{3.5} L)</math> operations on <math>O(L)</math> digit numbers, as compared to <math>O(n^6 L)</math> such operations for the ellipsoid algorithm. The runtime of Karmarkar's algorithm is thus 
:<math>O(n^{3.5} L^2 \cdot \log L \cdot \log \log L)</math>

using [[Schönhage–Strassen algorithm|FFT-based multiplication]] (see [[Big O notation]]).

Karmarkar's algorithm falls within the class of [[interior point method]]s: the current guess for the solution does not follow the boundary of the [[feasible set]] as in the [[simplex method]], but it moves through the interior of the feasible region, improving the approximation of the optimal solution by a definite fraction with every iteration, and converging to an optimal solution with rational data.<ref name="Strang">{{cite journal|last=Strang|first=Gilbert|authorlink=Gilbert Strang|title=Karmarkar's algorithm and its place in applied mathematics|journal=[[The Mathematical Intelligencer]]|date=1 June 1987|issn=0343-6993|pages=4–10|volume=9|number=2|doi=10.1007/BF03025891|mr=883185|ref=harv}}</ref>

== The algorithm ==
Consider a linear programming problem in matrix form:
{| 
| colspan="2" | maximize {{math|''c''<sup>T</sup>''x''}}
|-
| subject to
| {{math|''Ax'' ≤ ''b''}}.
|-
|}
Karmarkar's algorithm determines the next feasible direction toward optimality and scales back by a factor {{math|0 < γ ≤ 1}}. It is described in a number of sources.<ref>http://dl.acm.org/citation.cfm?id=808695</ref><ref>{{cite journal | doi=10.1007/BF02579150 | volume=4 |issue = 4| title=A new polynomial-time algorithm for linear programming | journal=Combinatorica | pages=373–395 | last1 = Karmarkar | first1 = N.|year = 1984}}</ref><ref>{{cite journal|doi=10.1002/j.1538-7305.1989.tb00316.x | volume=68 | issue=3 | title=Power Series Variants of Karmarkar-Type Algorithms | journal=AT&T Technical Journal | pages=20–36 | last1 = Karmarkar | first1 = Narendra K.| year=1989 }}</ref><ref>{{cite conference
 | last = Karmarkar | first = Narendra
 | contribution = An interior-point approach to NP-complete problems. I
 | doi = 10.1090/conm/114/1097880
 | mr = 1097880
 | pages = 297–308
 | publisher = American Mathematical Society | location = Providence, RI
 | series = Contemporary Mathematics
 | title = Mathematical developments arising from linear programming (Brunswick, ME, 1988)
 | volume = 114
 | year = 1990}}</ref><ref>{{cite conference
 | last = Karmarkar | first = Narendra
 | contribution = Riemannian geometry underlying interior-point methods for linear programming
 | doi = 10.1090/conm/114/1097865
 | mr = 1097865
 | pages = 51–75
 | publisher = American Mathematical Society | location = Providence, RI
 | series = Contemporary Mathematics
 | title = Mathematical developments arising from linear programming (Brunswick, ME, 1988)
 | volume = 114
 | year = 1990}}</ref><ref>Karmarkar N. K., Lagarias, J.C., Slutsman, L., and Wang, P., Power Series Variants of KarmarkarType Algorithm, AT & T technical Journal 68, No. 3, May/June (1989).</ref> Karmarkar also has extended the method<ref>Karmarkar, N.K., Interior Point Methods in Optimization, Proceedings of the Second International Conference on Industrial and Applied Mathematics, SIAM, pp. 160181 (1991)</ref><ref>Karmarkar, N. K. and Kamath, A. P., A continuous Approach to Deriving Upper Bounds in Quadratic Maximization Problems with Integer Constraints, Recent Advances in Global Optimization, pp. 125140, Princeton University Press (1992).</ref><ref>26.	Karmarkar, N. K., Thakur, S. A., An Interior Point Approach to a Tensor Optimisation Problem with Application to Upper Bounds in Integer Quadratic Optimization Problems, Proceedings of Second Conference on Integer Programming and Combinatorial Optimisation, (May 1992).</ref><ref>27.	Kamath, A., Karmarkar, N. K., A Continuous Method for Computing Bounds in Integer Quadratic Optimisation Problems, Journal of Global Optimization (1992).</ref> to solve problems with integer constraints and non-convex problems.<ref>Karmarkar, N. K., Beyond Convexity: New Perspectives in Computational Optimization. Springer Lecture Notes in Computer Science LNCS 6457, Dec 2010</ref>
{{algorithm-begin|name=Affine-Scaling}}

Since the actual algorithm is rather complicated, researchers looked for a more intuitive version of it, and in 1985 developed [[affine scaling]], a version of Karmarkar's algorithm that uses [[affine transformation]]s where Karmarkar used [[projective geometry|projective]] ones, only to realize four years later that they had rediscovered an algorithm published by [[Soviet Union|Soviet]] mathematician I. I. Dikin in 1967.<ref>{{cite conference
 | last1 = Vanderbei | first1 = R. J.
 | last2 = Lagarias | first2 = J. C.
 | contribution = I. I. Dikin's convergence result for the affine-scaling algorithm
 | doi = 10.1090/conm/114/1097868
 | mr = 1097868
 | pages = 109–119
 | publisher = American Mathematical Society | location = Providence, RI
 | series = Contemporary Mathematics
 | title = Mathematical developments arising from linear programming (Brunswick, ME, 1988)
 | volume = 114
 | year = 1990}}</ref> The affine-scaling method can be described succinctly as follows.<ref>{{cite journal
 | doi = 10.1007/BF01840454
 | author = Robert J. Vanderbei |author2=Meketon, Marc |author3=Freedman, Barry
 | year = 1986
 | title =  A Modification of Karmarkar's Linear Programming Algorithm 
 | journal = Algorithmica
 | volume = 1
 | issue = 1–4 | pages = 395–407 |url=https://www.princeton.edu/~rvdb/tex/myPapers/VanderbeiMeketonFreedman.pdf
| author-link = Robert J. Vanderbei }}</ref> The affine-scaling algorithm, while applicable to small scale problems, is not a polynomial time algorithm.{{Citation needed|date=February 2016}} 
   {{nowrap|Input:  A, b, c, <math>x^0</math>,}} ''stopping criterion'', {{mvar|&gamma;}}.

   {{nowrap|<math> k \leftarrow 0 </math>}}
   {{nowrap|'''do while''' ''stopping criterion'' '''not satisfied'''}}
      {{nowrap|<math>v^k \leftarrow b-Ax^k</math>}}
      {{nowrap|<math>D_v \leftarrow \operatorname{diag}(v_1^k,\ldots,v_m^k)</math>}}
      {{nowrap|<math>h_x\leftarrow (A^TD_v^{-2}A)^{-1}c</math>}}
      {{nowrap|<math>h_v\leftarrow -Ah_x</math>}}
      {{nowrap|'''if''' <math>h_v \ge 0</math> '''then'''}}
         '''return''' unbounded
      '''end if'''
      {{nowrap|<math>\alpha \leftarrow \gamma\cdot \min\{-v_i^k/(h_v)_i \,\,|\,\, (h_v)_i < 0,\, i=1,\ldots,m\}</math>}}
      {{nowrap|<math>x^{k+1}\leftarrow x^k + \alpha h_x</math>}}
      {{nowrap|<math>k\leftarrow k+1</math>}}
   '''end do'''
{{algorithm-end}}

== Example ==
[[File:karmarkar.svg|thumb|200px|right|Example solution]]
Consider the linear program
:<math> \begin{array}{lrclr}
  \text{maximize}
  &
  x_1
  +
  x_2
  \\
  \text{subject to}
  &
  2p x_1
  + 
  x_2 
  & \leq & p^2+1, & p=0.0, 0.1, 0.2,\ldots, 0.9, 1.0.
\end{array}
</math>
That is, there are 2 variables <math>x_1, x_2</math> and 11 constraints associated with varying values of <math>p</math>. This figure shows each iteration of the algorithm as red circle points. The constraints are shown as blue lines.

== Patent controversy ==
At the time he invented the algorithm, Karmarkar was employed by [[IBM]] as a postdoctoral fellow in the [[IBM San Jose Research Laboratory]] in California. On August 11, 1983 he gave a seminar at [[Stanford University]] explaining the algorithm, with his affiliation still listed as IBM. By the fall of 1983 Karmarkar started to work at [[AT&T]] and submitted his paper to the 1984 ACM [[Symposium on Theory of Computing]] (STOC, held April 30 - May 2, 1984) stating [[AT&T Bell Laboratories]] as his affiliation.<ref>{{citation|title=Karmarkar Algorithm|url=http://researcher.watson.ibm.com/researcher/view_page.php?id=6900|publisher=IBM Research|accessdate=2016-06-01}}</ref> After applying the algorithm to optimizing AT&T's telephone network,<ref>Sinha L.P., Freedman, B. A., Karmarkar, N. K., Putcha, A., and Ramakrishnan K.G., Overseas Network Planning, Proceedings of the Third International Network Planning Symposium, NETWORKS' 86, Tarpon Springs, Florida (June 1986).</ref> they realized that his invention could be of practical importance.  In April 1985, AT&T promptly applied for a patent on Karmarkar's algorithm.

The patent became more fuel for the ongoing controversy over the issue of [[software patent]]s.<ref name="kolata">
{{cite news
 | first = Gina
 | last = Kolata
 | title = IDEAS & TRENDS; Mathematicians Are Troubled by Claims on Their Recipes
 | url = https://query.nytimes.com/gst/fullpage.html?res=950DEFD61038F931A25750C0A96F948260
 | work = [[The New York Times]]
 | date = 1989-03-12
}}</ref>  This left many mathematicians uneasy, such as [[Ronald Rivest]] (himself one of the holders of the patent on the [[RSA (algorithm)|RSA]] algorithm), who expressed the opinion that research proceeded on the basis that algorithms should be free. Even before the patent was actually granted, it was argued that there might have been [[prior art]] that was applicable.<ref name="saltzman">[https://www.cs.cmu.edu/afs/cs.cmu.edu/academic/class/15850c-s96/www/interiorpoint.txt Various posts by Matthew Saltzman, Clemson University]</ref> Mathematicians who specialized in [[numerical analysis]], including [[Philip Gill]] and others, claimed that Karmarkar's algorithm is equivalent to a [[projected Newton barrier method]] with a logarithmic [[barrier function]], if the parameters are chosen suitably.<ref>
{{cite journal
 | last = Gill
 | first = Philip E.
 | last2 = Murray |first2= Walter |last3= Saunders |first3= Michael A. |last4= Tomlin |first4= J. A. |last5= Wright |first5= Margaret H.
 | year = 1986
 | title =  On projected Newton barrier methods for linear programming and an equivalence to Karmarkar's projective method 
 | journal = Mathematical Programming
 | volume = 36
 | issue = 2
 | pages = 183–209
 | doi = 10.1007/BF02592025
}}</ref>  Legal scholar Andrew Chin opines that Gill's argument was flawed, insofar as the method they describe does not constitute an "algorithm", since it requires choices of parameters that don't follow from the internal logic of the method, but rely on external guidance, essentially from Karmarkar's algorithm.<ref name="chin">
{{cite journal
 | author = Andrew Chin
 | year = 2009
 | title =  On Abstraction and Equivalence in Software Patent Doctrine: A Response to Bessen, Meurer and Klemens
 | journal = Journal of Intellectual Property Law
 | volume = 16
 | pages = 214–223
 | url = http://andrewchin.com/chin/scholarship/abstraction-equivalence.pdf
}}</ref> Furthermore, Karmarkar's contributions are considered far from obvious in light of all prior work, including Fiacco-McCormick, Gill and others cited by Saltzman.<ref name="chin"/><ref name="paley">Mark A. Paley  (1995). "The Karmarkar Patent:  Why Congress Should "Open the Door" to Algorithms as Patentable Subject Matter". 22 COMPUTER  L. REP. 7</ref><ref name="wright">{{cite journal
 | author = Margaret H. Wright
 | year = 2004
 | title =  The Interior-Point Revolution in Optimization: History, Recent Developments, and Lasting Consequences
 | journal = Bulletin of the American Mathematical Society
 | volume = 42
 | pages = 39–56
 | url = http://www.ams.org/journals/bull/2005-42-01/S0273-0979-04-01040-7/S0273-0979-04-01040-7.pdf
 | doi=10.1090/S0273-0979-04-01040-7
}}</ref>  The patent was debated in the U.S. Senate and granted in recognition of the essential originality of Karmarkar's work, as {{US patent|4744028}}: "Methods and apparatus for efficient resource allocation" in May 1988.

AT&T designed a [[vector processor|vector]] [[multi-processor]] computer system specifically to run Karmarkar's algorithm, calling the resulting combination of hardware and software KORBX,<ref>
{{cite journal
 | author = Marc S. Meketon
 |author2=Y.C. Cheng |author3=D.J. Houck |author4=J.M.Liu |author5=L. Slutsman |author6-link=Robert J. Vanderbei |author6=Robert J. Vanderbei |author7=P. Wang
 | year = 1989
 | title =  The AT&T KORBX System
 | journal = AT&T Technical Journal
 | volume = 68
 |issue=3 | pages = 7–19
|doi=10.1002/j.1538-7305.1989.tb00315.x }}</ref> and marketed this system at a price of US$8.9 million.<ref>{{cite news |title=AT&T markets problem solver, based on math whiz's find, for $8.9 million |newspaper=Wall Street Journal |date=15 August 1988 |first=Roger |last=Lowenstein |url=http://orfe.princeton.edu/~rvdb/307/lectures/lec19.pdf}}</ref><ref>{{cite news |title=Big A.T.&T. Computer for Complexities |first=John |last=Markoff |date=13 August 1988 |url=https://www.nytimes.com/1988/08/13/business/big-at-t-computer-for-complexities.html}}</ref> Its first customer was the [[United States Department of Defense|Pentagon]].<ref>{{Cite web|url=https://apnews.com/8a376783cd62cdf141de700a7c948f61|title=Military Is First Announced Customer Of AT&T Software|website=AP NEWS|access-date=2019-06-11}}</ref><ref>{{Cite book | chapter-url=http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=70419&url=http%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%3D70419 | doi=10.1109/CDC.1989.70419| chapter=Using KORBX for military airlift applications| title=Proceedings of the 28th IEEE Conference on Decision and Control| pages=1603–1605| year=1989| last1=Kennington| first1=J.L.}}</ref>

Opponents of software patents have further argued that the patents ruined the positive interaction cycles that previously characterized the relationship between researchers in linear programming and industry, and specifically it isolated Karmarkar himself from the network of mathematical researchers in his field.<ref>{{Cite web
|url=http://eupat.ffii.org/papers/konno95/index.ja.html
|title=今野浩: カーマーカー特許とソフトウェア – 数学は 特許に なるか (Konno Hiroshi: The Kamarkar Patent and Software – Has Mathematics Become Patentable?)
|publisher=[[Foundation for a Free Information Infrastructure|FFII]]
|accessdate=2008-06-27}}</ref>

The patent itself expired in April 2006, and the algorithm is presently in the [[public domain]].

The [[United States Supreme Court]] has held that mathematics cannot be patented in ''[[Gottschalk v. Benson]]'',<ref>409 U.S. 63 (1972). The case concerned an algorithm for converting binary-coded decimal numerals to pure binary.</ref> In that case, the Court first addressed whether computer algorithms could be patented and it held that they could not because the patent system does not protect ideas and similar abstractions. In ''[[Diamond v. Diehr]]'',<ref>450 U.S. 175 (1981).</ref> the Supreme Court stated, "A mathematical formula as such is not accorded the protection of our patent laws, and this principle cannot be circumvented by attempting to limit the use of the formula to a particular technological environment.<ref>450 U.S. at 191. See also ''[[Parker v. Flook]]'', 437 U.S. 584, 585 (1978) ("the discovery of a novel and useful mathematical formula may not be patented").</ref> In ''[[Mayo Collaborative Services v. Prometheus Labs., Inc.]]'',<ref>566 U.S. __, 132 S. Ct. 1289 (2012).</ref> the Supreme Court explained further that "simply implementing a mathematical principle on a physical machine, namely a computer, [i]s not a patentable application of that principle."<ref>Accord ''[[Alice Corp. v. CLS Bank Int’l]]'',  573 U.S. __, 134 S. Ct. 2347  (2014).</ref>

== References ==

* {{cite journal | last1 = Adler | first1 = Ilan | last2 = Karmarkar | first2 = Narendra | last3 = Resende | first3 = Mauricio G.C. | last4 = Veiga | first4 = Geraldo | year = 1989 | title = An Implementation of Karmarkar's Algorithm for Linear Programming | url = | journal = Mathematical Programming | volume = 44 | issue = 1–3| pages = 297–335 | doi=10.1007/bf01587095}}
* Narendra Karmarkar (1984). "[https://web.archive.org/web/20131228145520/http://retis.sssup.it/~bini/teaching/optim2010/karmarkar.pdf A New Polynomial Time Algorithm for Linear Programming]", ''[[Combinatorica]]'', Vol '''4''', nr. 4, p.&nbsp;373&ndash;395.

{{reflist|30em}}

{{Optimization algorithms|convex}}

{{DEFAULTSORT:Karmarkar's Algorithm}}
[[Category:Optimization algorithms and methods]]
[[Category:Articles with example pseudocode]]
[[Category:Software patent law]]
[[Category:Linear programming]]